[DISCUSSION] where download gradle-wrapper.jar ?

[Nicolas Malin-2]

Hello,

With the commit r1861766 on trunk[1], I removed gradle-wrapper.jar from
source (OFBIZ-10145[2]) and now we download it at the first time when
you run 'gradlew'. It's fast download (55ko) so normally transparent

You can try yourself after update your trunk, it's esay :) .

Jacques and me have a different vision on where we should download the
jar and before continue to propagate the deleting on stable branches, I
like to get your opinion.

Jacques suggests to store the gradle-wrapper.jar on own tools repository
source[3] and download through viewvc with one wrapper by stable branch.
I suggest to download it from gradle community on github and store the
release to download in own code source[4]

Your opinion ? or other idea ?


Nicolas

[1]
https://svn.apache.org/viewvc?diff_format=h&view=revision&revision=1861766
[2] https://issues.apache.org/jira/browse/OFBIZ-10145
[3]
https://svn.apache.org/repos/asf/ofbiz/tools/Buildbot/Gradle/Wrapper/trunk/
[4] https://github.com/gradle/gradle/tree/v5.0.0/gradle/wrapper

--
logoNrd <https://nereide.fr/>
        Nicolas Malin
The apache way <http://theapacheway.com/> : *Charity* Apache’s mission
is providing software for the public good.
[hidden email]
8 rue des Déportés 37000 TOURS, 02 47 50 30 54

Apache OFBiz <http://ofbiz.apache.org/>|The Apache Way
<http://theapacheway.com/>|réseau LE <http://www.libre-entreprise.org/>

[Jacques Le Roux]

Thanks Nicolas,

The reason I finally decided to get this way is commented here: https://s.apache.org/Vsy3

In few words, we can no longer find the 2.13 wrapper version at https://github.com/gradle/gradle/tree/v2.13/gradle/wrapper like we still can for 3.2.1
and 5.0

Our stable release relies on it. It's a blocker. And I fear the same could happen in future for newer OFBiz releases. That's why I prefer to keep
copies in our repo.

BTW, Swapnil raised a new issue yesterday. With the current solution what to do for our users when we upgrade Gradle on the trunk?

I suggested a simple solution, so better to read at least the last comments, thanks.

As always, all ideas are welcome

Jacques

Le 22/06/2019 à 16:53, Nicolas Malin a écrit :

> Hello,
>
> With the commit r1861766 on trunk[1], I removed gradle-wrapper.jar from source (OFBIZ-10145[2]) and now we download it at the first time when you
> run 'gradlew'. It's fast download (55ko) so normally transparent
>
> You can try yourself after update your trunk, it's esay :) .
>
> Jacques and me have a different vision on where we should download the jar and before continue to propagate the deleting on stable branches, I like
> to get your opinion.
>
> Jacques suggests to store the gradle-wrapper.jar on own tools repository source[3] and download through viewvc with one wrapper by stable branch.
> I suggest to download it from gradle community on github and store the release to download in own code source[4]
>
> Your opinion ? or other idea ?
>
>
> Nicolas
>
> [1] https://svn.apache.org/viewvc?diff_format=h&view=revision&revision=1861766
> [2] https://issues.apache.org/jira/browse/OFBIZ-10145
> [3] https://svn.apache.org/repos/asf/ofbiz/tools/Buildbot/Gradle/Wrapper/trunk/
> [4] https://github.com/gradle/gradle/tree/v5.0.0/gradle/wrapper
>

[Mathieu Lirzin]

Hello,

Nicolas Malin <[hidden email]> writes:

> With the commit r1861766 on trunk[1], I removed gradle-wrapper.jar
> from source (OFBIZ-10145[2]) and now we download it at the first time
> when you run 'gradlew'. It's fast download (55ko) so normally
> transparent
>
> You can try yourself after update your trunk, it's esay :) .

The “gradlew-wrapper.jar” should not have been removed from ‘trunk’ in
the first place. Sorry for repeating myself but the requirements of not
distributing this jar is concerning only the *releases archives* not the
*repositories* as stated in LEGAL-288 [1].

For convenience it has been proposed to patch the “gradlew” script
included in *release archive* instead of providing only the instructions
telling the user how to download the wrapper manually.

The act of patching the “gradlew” script should be done at release time,
and we should not commit the patched script in our VCS repository since
as Swapnil [2] explained it doesn't play well with the standard way of
upgrading Gradle:

   $ ./gradlew wrapper --gradle-version=5.4.1 --distribution-type=bin

I don't have an opinion regarding how the patching process should be
documented in the release instructions.

[1] https://issues.apache.org/jira/browse/LEGAL-288?focusedCommentId=15980830&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-15980830
[2] https://issues.apache.org/jira/browse/OFBIZ-10145?focusedCommentId=16869382&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16869382

> Jacques and me have a different vision on where we should download the
> jar and before continue to propagate the deleting on stable branches,
> I like to get your opinion.

Can you add “gradlew-wrapper.jar” back in ‘trunk’ instead? :-)

> Jacques suggests to store the gradle-wrapper.jar on own tools
> repository source[3] and download through viewvc with one wrapper by
> stable branch.
> I suggest to download it from gradle community on github and store the
> release to download in own code source[4]
>
> Your opinion ? or other idea ?

IMO storing Gradle binary releases in our own infrastructure is too much
trouble. If we don't trust Gradle binaries to continue to be available
in the future, we maybe should use another build system instead of
keeping trying to keep our copies of its binary releases. No?

--
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37

[Mathieu Lirzin]

Mathieu Lirzin <[hidden email]> writes:

> Nicolas Malin <[hidden email]> writes:
>
>> Jacques suggests to store the gradle-wrapper.jar on own tools
>> repository source[3] and download through viewvc with one wrapper by
>> stable branch.
>> I suggest to download it from gradle community on github and store the
>> release to download in own code source[4]
>>
>> Your opinion ? or other idea ?
>
> IMO storing Gradle binary releases in our own infrastructure is too much
> trouble. If we don't trust Gradle binaries to continue to be available
> in the future, we maybe should use another build system instead of
> keeping trying to keep our copies of its binary releases. No?

I overlooked that we were not speaking of downloading the binary
releases, but only of the “gradle-wrapper.jar” which should already be
avaibable in our VCS branches in order to comply with Gradle wrapping
recommendations as I advocated in my previous mail.

Given that each release archive can point to its corresponding
repository branch to find its associated “gradle-wrapper.jar”, it is
basically free in term of maintenance. Then I agree with Jacques option.

Sorry for the confusion.

--
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37

[taher]

I think the gradle-wrapper.jar should be maintained inside the code
base as an exceptional thing, otherwise it would be quite painful to
administer the system.

The gradle wrapper is difficult to download and obtain, and is usually
provided using a gradle command (installed gradle on the platform. So
it's best to just keep it in.

On Sun, Jun 23, 2019 at 4:58 PM Mathieu Lirzin
<[hidden email]> wrote:

>
> Mathieu Lirzin <[hidden email]> writes:
>
> > Nicolas Malin <[hidden email]> writes:
> >
> >> Jacques suggests to store the gradle-wrapper.jar on own tools
> >> repository source[3] and download through viewvc with one wrapper by
> >> stable branch.
> >> I suggest to download it from gradle community on github and store the
> >> release to download in own code source[4]
> >>
> >> Your opinion ? or other idea ?
> >
> > IMO storing Gradle binary releases in our own infrastructure is too much
> > trouble. If we don't trust Gradle binaries to continue to be available
> > in the future, we maybe should use another build system instead of
> > keeping trying to keep our copies of its binary releases. No?
>
> I overlooked that we were not speaking of downloading the binary
> releases, but only of the “gradle-wrapper.jar” which should already be
> avaibable in our VCS branches in order to comply with Gradle wrapping
> recommendations as I advocated in my previous mail.
>
> Given that each release archive can point to its corresponding
> repository branch to find its associated “gradle-wrapper.jar”, it is
> basically free in term of maintenance. Then I agree with Jacques option.
>
> Sorry for the confusion.
>
> --
> Mathieu Lirzin
> GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37

[Jacques Le Roux]

Yes, as long as we don't release it there are no problems with that.

Jacques

Le 23/06/2019 à 17:38, Taher Alkhateeb a écrit :

> I think the gradle-wrapper.jar should be maintained inside the code
> base as an exceptional thing, otherwise it would be quite painful to
> administer the system.
>
> The gradle wrapper is difficult to download and obtain, and is usually
> provided using a gradle command (installed gradle on the platform. So
> it's best to just keep it in.
>
> On Sun, Jun 23, 2019 at 4:58 PM Mathieu Lirzin
> <[hidden email]> wrote:
>> Mathieu Lirzin <[hidden email]> writes:
>>
>>> Nicolas Malin <[hidden email]> writes:
>>>
>>>> Jacques suggests to store the gradle-wrapper.jar on own tools
>>>> repository source[3] and download through viewvc with one wrapper by
>>>> stable branch.
>>>> I suggest to download it from gradle community on github and store the
>>>> release to download in own code source[4]
>>>>
>>>> Your opinion ? or other idea ?
>>> IMO storing Gradle binary releases in our own infrastructure is too much
>>> trouble. If we don't trust Gradle binaries to continue to be available
>>> in the future, we maybe should use another build system instead of
>>> keeping trying to keep our copies of its binary releases. No?
>> I overlooked that we were not speaking of downloading the binary
>> releases, but only of the “gradle-wrapper.jar” which should already be
>> avaibable in our VCS branches in order to comply with Gradle wrapping
>> recommendations as I advocated in my previous mail.
>>
>> Given that each release archive can point to its corresponding
>> repository branch to find its associated “gradle-wrapper.jar”, it is
>> basically free in term of maintenance. Then I agree with Jacques option.
>>
>> Sorry for the confusion.
>>
>> --
>> Mathieu Lirzin
>> GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37

[Nicolas Malin-2]

My vision is to have the code base near the building package as
possible. If we keep gradle-wrapper.jar for developing comfort, it's a
risk for me to generate an issue on release package due to not enough
test after package creation.

> Le 23/06/2019 à 17:38, Taher Alkhateeb a écrit :
>> [...]
>> The gradle wrapper is difficult to download and obtain, and is usually
>> provided using a gradle command (installed gradle on the platform. So
>> it's best to just keep it in.

Taher do you show any difficulties with the trunk to resolve
automatically and download the wrapper ?


Otherwise, I have a problem with the resolution on gradle-wrapper based
only on branches, with that the wrapper resolved would be always the up
to date related to branch code base. It's nice for demo, buildbot and
developer but open a potential problem for package.

16.05 works with gradle 3.2.1 for any reason we need to update the
wrapper to 4.0.0, 16.06 works with it but all previous package has been
a potential incompatibility.We can decide to say it's not own problem
however I prefer to offer a solution more flexible like store the
wrapper by version : ofbiz/tools/gradle/wrapper/v3.2.1/* as the gradle
community works. Also possibility to call wrapper verion by wrapper it
self, I really une preference to resolve direclty the wrapper on the
good version.

On second time I prefer to resolve the wrapper from the origin community
(keep resolution near source code, have always source origin) but after
different remarks maybe we can couple gradle community and own
resolution form own repository as fallback.

Nicolas

>>
>> On Sun, Jun 23, 2019 at 4:58 PM Mathieu Lirzin
>> <[hidden email]> wrote:
>>> Mathieu Lirzin <[hidden email]> writes:
>>>
>>>> Nicolas Malin <[hidden email]> writes:
>>>>
>>>>> Jacques suggests to store the  on own tools
>>>>> repository source[3] and download through viewvc with one wrapper by
>>>>> stable branch.
>>>>> I suggest to download it from gradle community on github and store
>>>>> the
>>>>> release to download in own code source[4]
>>>>>
>>>>> Your opinion ? or other idea ?
>>>> IMO storing Gradle binary releases in our own infrastructure is too
>>>> much
>>>> trouble. If we don't trust Gradle binaries to continue to be available
>>>> in the future, we maybe should use another build system instead of
>>>> keeping trying to keep our copies of its binary releases. No?
>>> I overlooked that we were not speaking of downloading the binary
>>> releases, but only of the “gradle-wrapper.jar” which should already be
>>> avaibable in our VCS branches in order to comply with Gradle wrapping
>>> recommendations as I advocated in my previous mail.
>>>
>>> Given that each release archive can point to its corresponding
>>> repository branch to find its associated “gradle-wrapper.jar”, it is
>>> basically free in term of maintenance. Then I agree with Jacques
>>> option.
>>>
>>> Sorry for the confusion.
>>>
>>> --
>>> Mathieu Lirzin
>>> GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37
>
>

[taher]

You can go ahead and give it a try, however the main problem you might
face would perhaps be the complex logic that you need to add to
"gradlew" and "gradlew.bat" scripts to download the jar if it does not
exist. Now that wrapper jar is difficult to obtain because I think you
can only get it from inside the compressed archive of a certain
distribution.

So think about it, the logic in these two scripts (bash & windows
batch) should have the following logic in it:
1 - download the correct compressed archive binary
2 - uncompress it
3 - copy the wrapper jar to the correct location

The first two items above require installation of certain tools on the
operating system like curl, wget, zip, etc ... which means they won't
work out of the box and requirements to run ofbiz would go beyond
simply installing java. That's what I meant by difficult and complex.

Another downsize to heavy customization of these scripts is that the
upgrade process of the wrapper in the new version becomes more
involved because these scripts are auto generated by gradle, and so
merging these changes into your custom logic might be more work.

Now to reduce that complexity we might switch from a batch script to a
powershell script instead, but that would mean converting the entire
script to powershell and also means powershell is a necessity on any
Microsoft platform.

So if we bypass the wrapper script as an exceptional thing, it would
be the easiest way to move forward and I think this was discussed
elsewhere in the ASF before (perhaps LEGAL).

To summarize, I'm not objecting to doing it if you would like to push
in that direction, but it would perhaps be annoying and complex to
implement.

On Tue, Jun 25, 2019 at 12:49 PM Nicolas Malin <[hidden email]> wrote:

>
> My vision is to have the code base near the building package as
> possible. If we keep gradle-wrapper.jar for developing comfort, it's a
> risk for me to generate an issue on release package due to not enough
> test after package creation.
>
> > Le 23/06/2019 à 17:38, Taher Alkhateeb a écrit :
> >> [...]
> >> The gradle wrapper is difficult to download and obtain, and is usually
> >> provided using a gradle command (installed gradle on the platform. So
> >> it's best to just keep it in.
>
> Taher do you show any difficulties with the trunk to resolve
> automatically and download the wrapper ?
>
>
> Otherwise, I have a problem with the resolution on gradle-wrapper based
> only on branches, with that the wrapper resolved would be always the up
> to date related to branch code base. It's nice for demo, buildbot and
> developer but open a potential problem for package.
>
> 16.05 works with gradle 3.2.1 for any reason we need to update the
> wrapper to 4.0.0, 16.06 works with it but all previous package has been
> a potential incompatibility.We can decide to say it's not own problem
> however I prefer to offer a solution more flexible like store the
> wrapper by version : ofbiz/tools/gradle/wrapper/v3.2.1/* as the gradle
> community works. Also possibility to call wrapper verion by wrapper it
> self, I really une preference to resolve direclty the wrapper on the
> good version.
>
> On second time I prefer to resolve the wrapper from the origin community
> (keep resolution near source code, have always source origin) but after
> different remarks maybe we can couple gradle community and own
> resolution form own repository as fallback.
>
> Nicolas
>
>
> >>
> >> On Sun, Jun 23, 2019 at 4:58 PM Mathieu Lirzin
> >> <[hidden email]> wrote:
> >>> Mathieu Lirzin <[hidden email]> writes:
> >>>
> >>>> Nicolas Malin <[hidden email]> writes:
> >>>>
> >>>>> Jacques suggests to store the  on own tools
> >>>>> repository source[3] and download through viewvc with one wrapper by
> >>>>> stable branch.
> >>>>> I suggest to download it from gradle community on github and store
> >>>>> the
> >>>>> release to download in own code source[4]
> >>>>>
> >>>>> Your opinion ? or other idea ?
> >>>> IMO storing Gradle binary releases in our own infrastructure is too
> >>>> much
> >>>> trouble. If we don't trust Gradle binaries to continue to be available
> >>>> in the future, we maybe should use another build system instead of
> >>>> keeping trying to keep our copies of its binary releases. No?
> >>> I overlooked that we were not speaking of downloading the binary
> >>> releases, but only of the “gradle-wrapper.jar” which should already be
> >>> avaibable in our VCS branches in order to comply with Gradle wrapping
> >>> recommendations as I advocated in my previous mail.
> >>>
> >>> Given that each release archive can point to its corresponding
> >>> repository branch to find its associated “gradle-wrapper.jar”, it is
> >>> basically free in term of maintenance. Then I agree with Jacques
> >>> option.
> >>>
> >>> Sorry for the confusion.
> >>>
> >>> --
> >>> Mathieu Lirzin
> >>> GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37
> >
> >

[Jacques Le Roux]

Hi Nicolas,

Inline....

Le 25/06/2019 à 11:49, Nicolas Malin a écrit :
> My vision is to have the code base near the building package as possible. If we keep gradle-wrapper.jar for developing comfort, it's a risk for me
> to generate an issue on release package due to not enough test after package creation.

Normally the wrapper version in tools corresponds to an OFBiz package, then no worries at all.
I'd address below the unlikely  issue of multiple wrapper versions for an OFBiz package.
If it's your concern, you have a point with that, we missed it so far and I thank you for pushing us to think more. BTW same compliment to Swapnil :)
I can't foresee other possible problems, do you have something else in mind?


> Otherwise, I have a problem with the resolution on gradle-wrapper based only on branches, with that the wrapper resolved would be always the up to
> date related to branch code base. It's nice for demo, buildbot and developer but open a potential problem for package.
> 16.05 works with gradle 3.2.1 for any reason we need to update the wrapper to 4.0.0, 16.06 works with it but all previous package has been a
> potential incompatibility.We can decide to say it's not own problem however I prefer to offer a solution more flexible like store the wrapper by
> version : ofbiz/tools/gradle/wrapper/v3.2.1/* as the gradle community works. Also possibility to call wrapper verion by wrapper it self, I really
> une preference to resolve direclty the wrapper on the good version.
This is indeed a potential concern if a Gradle version has a (high) vulnerability. We would then (absolutely) need to upgrade Gradle and release new
OFBiz packages for supported branches.This said, it would be indeed an issue in the future for users still using a "not longer supported" OFBiz
released package. Do you see other reasons for discrepancy between the wrapper in tools and an OFBiz package?

I say potential because a high Gradle vulnerability never happened so far and even a low Gradle vulnerability is a very rare thing.  It happened once
2+ years ago, only in Gradle 2.12 (04/2016), that we never used[1].  Of course this does not mean that another vulnerability could not happen in future...

Currently the idea is to keep a version of the wrapper by branch as it's done in tools[2]. And already used in Buildbot[3], even if there for the
moment it only overwrites the same version by branch.

I suggest that we not only keep a version by branch but, if necessary, also by versions used by a release package. So, for instance, if we need to
update 16.11.06 to use a newer version of the wrapper we would keep this version in tools as[4], would create scripts specifically for it and embed
them in the package. Using ofbiz/tools/gradle/wrapper/v3.2.1/*, as you suggest, fits also with me :)

[1] https://www.cvedetails.com/vulnerability-list/vendor_id-16071/Gradle.html (Gradle Enterprise is another thing and not OFBiz responsibility)
[2] https://svn.apache.org/repos/asf/ofbiz/tools/Buildbot/Gradle/Wrapper
[3] https://svn.apache.org/repos/infra/infrastructure/buildbot/aegis/buildmaster/master1/projects/ofbiz.conf
[4] https://svn.apache.org/repos/asf/ofbiz/tools/Buildbot/Gradle/Wrapper/16.11.06


>
> On second time I prefer to resolve the wrapper from the origin community (keep resolution near source code, have always source origin) but after
> different remarks maybe we can couple gradle community and own resolution form own repository as fallback.
>
> Nicolas
>
I think I mostly rephrased your proposition to be sure we are on the same page, and for people to get a clear idea of the situation.

I see only one future problem getting wrappers directly from Gradle community in released OFBiz packages: when it become impossible (eg v2.13). We
don't know when it will happen, we just know that it will happen! Having them saved in our repo is peace of mind...

Again thanks for the discussion :)

Jacques

>>>
>>> On Sun, Jun 23, 2019 at 4:58 PM Mathieu Lirzin
>>> <[hidden email]> wrote:
>>>> Mathieu Lirzin <[hidden email]> writes:
>>>>
>>>>> Nicolas Malin <[hidden email]> writes:
>>>>>
>>>>>> Jacques suggests to store the  on own tools
>>>>>> repository source[3] and download through viewvc with one wrapper by
>>>>>> stable branch.
>>>>>> I suggest to download it from gradle community on github and store the
>>>>>> release to download in own code source[4]
>>>>>>
>>>>>> Your opinion ? or other idea ?
>>>>> IMO storing Gradle binary releases in our own infrastructure is too much
>>>>> trouble. If we don't trust Gradle binaries to continue to be available
>>>>> in the future, we maybe should use another build system instead of
>>>>> keeping trying to keep our copies of its binary releases. No?
>>>> I overlooked that we were not speaking of downloading the binary
>>>> releases, but only of the “gradle-wrapper.jar” which should already be
>>>> avaibable in our VCS branches in order to comply with Gradle wrapping
>>>> recommendations as I advocated in my previous mail.
>>>>
>>>> Given that each release archive can point to its corresponding
>>>> repository branch to find its associated “gradle-wrapper.jar”, it is
>>>> basically free in term of maintenance. Then I agree with Jacques option.
>>>>
>>>> Sorry for the confusion.
>>>>
>>>> --
>>>> Mathieu Lirzin
>>>> GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37
>>
>>
>

[Jacques Le Roux]

Le 26/06/2019 à 16:17, Jacques Le Roux a écrit :
> Using ofbiz/tools/gradle/wrapper/v3.2.1/*, as you suggest, fits also with me :)
It's actually a (slightly) better solution, because then there no duplication (we speak about Kb here). Though it's maybe less clear than named
revisions. Anyway it's the community to decide

Jacques

[Jacques Le Roux]

Le 25/06/2019 à 13:53, Taher Alkhateeb a écrit :
> powershell is a necessity on any
> Microsoft platform.

Hi Taher,

Actually we need Powershell anyway. Dos script does not allow to get files from Internet. We need Invoke-WebRequest for that (aliased as wget in[1])

This is not an issue. Invoke-WebRequest was put in with Powershell 3.0 which is available in Windows 7[2] and anyway you can still follow[3]

[1] https://svn.apache.org/repos/asf/ofbiz/ofbiz-framework/trunk/gradle/init-gradle-wrapper.ps1
[2] https://en.wikipedia.org/wiki/PowerShell#PowerShell_3.0
[3] https://codehollow.com/2016/02/invoke-webrequests-via-powershell/

HTH

Jacques

[Nicolas Malin-2]

I would had tend to be agree with you with the first step done.

At this time, only [curl,wget] is necessary, and download realease at
the fisrt gradlew command call.

We take the jar, put it directly at the good place and continue the job.

If the person haven't [curl,wget], it's in general more easier to load
it instead java jre, and if no Internet is present this solve the
problem because he can't continue anyway to download OFBiz's dependencies :)

Nicolas

On 25/06/2019 13:53, Taher Alkhateeb wrote:

> You can go ahead and give it a try, however the main problem you might
> face would perhaps be the complex logic that you need to add to
> "gradlew" and "gradlew.bat" scripts to download the jar if it does not
> exist. Now that wrapper jar is difficult to obtain because I think you
> can only get it from inside the compressed archive of a certain
> distribution.
>
> So think about it, the logic in these two scripts (bash & windows
> batch) should have the following logic in it:
> 1 - download the correct compressed archive binary
> 2 - uncompress it
> 3 - copy the wrapper jar to the correct location
>
> The first two items above require installation of certain tools on the
> operating system like curl, wget, zip, etc ... which means they won't
> work out of the box and requirements to run ofbiz would go beyond
> simply installing java. That's what I meant by difficult and complex.
>
> Another downsize to heavy customization of these scripts is that the
> upgrade process of the wrapper in the new version becomes more
> involved because these scripts are auto generated by gradle, and so
> merging these changes into your custom logic might be more work.
>
> Now to reduce that complexity we might switch from a batch script to a
> powershell script instead, but that would mean converting the entire
> script to powershell and also means powershell is a necessity on any
> Microsoft platform.
>
> So if we bypass the wrapper script as an exceptional thing, it would
> be the easiest way to move forward and I think this was discussed
> elsewhere in the ASF before (perhaps LEGAL).
>
> To summarize, I'm not objecting to doing it if you would like to push
> in that direction, but it would perhaps be annoying and complex to
> implement.
>
> On Tue, Jun 25, 2019 at 12:49 PM Nicolas Malin <[hidden email]> wrote:
>> My vision is to have the code base near the building package as
>> possible. If we keep gradle-wrapper.jar for developing comfort, it's a
>> risk for me to generate an issue on release package due to not enough
>> test after package creation.
>>
>>> Le 23/06/2019 à 17:38, Taher Alkhateeb a écrit :
>>>> [...]
>>>> The gradle wrapper is difficult to download and obtain, and is usually
>>>> provided using a gradle command (installed gradle on the platform. So
>>>> it's best to just keep it in.
>> Taher do you show any difficulties with the trunk to resolve
>> automatically and download the wrapper ?
>>
>>
>> Otherwise, I have a problem with the resolution on gradle-wrapper based
>> only on branches, with that the wrapper resolved would be always the up
>> to date related to branch code base. It's nice for demo, buildbot and
>> developer but open a potential problem for package.
>>
>> 16.05 works with gradle 3.2.1 for any reason we need to update the
>> wrapper to 4.0.0, 16.06 works with it but all previous package has been
>> a potential incompatibility.We can decide to say it's not own problem
>> however I prefer to offer a solution more flexible like store the
>> wrapper by version : ofbiz/tools/gradle/wrapper/v3.2.1/* as the gradle
>> community works. Also possibility to call wrapper verion by wrapper it
>> self, I really une preference to resolve direclty the wrapper on the
>> good version.
>>
>> On second time I prefer to resolve the wrapper from the origin community
>> (keep resolution near source code, have always source origin) but after
>> different remarks maybe we can couple gradle community and own
>> resolution form own repository as fallback.
>>
>> Nicolas
>>
>>
>>>> On Sun, Jun 23, 2019 at 4:58 PM Mathieu Lirzin
>>>> <[hidden email]> wrote:
>>>>> Mathieu Lirzin <[hidden email]> writes:
>>>>>
>>>>>> Nicolas Malin <[hidden email]> writes:
>>>>>>
>>>>>>> Jacques suggests to store the  on own tools
>>>>>>> repository source[3] and download through viewvc with one wrapper by
>>>>>>> stable branch.
>>>>>>> I suggest to download it from gradle community on github and store
>>>>>>> the
>>>>>>> release to download in own code source[4]
>>>>>>>
>>>>>>> Your opinion ? or other idea ?
>>>>>> IMO storing Gradle binary releases in our own infrastructure is too
>>>>>> much
>>>>>> trouble. If we don't trust Gradle binaries to continue to be available
>>>>>> in the future, we maybe should use another build system instead of
>>>>>> keeping trying to keep our copies of its binary releases. No?
>>>>> I overlooked that we were not speaking of downloading the binary
>>>>> releases, but only of the “gradle-wrapper.jar” which should already be
>>>>> avaibable in our VCS branches in order to comply with Gradle wrapping
>>>>> recommendations as I advocated in my previous mail.
>>>>>
>>>>> Given that each release archive can point to its corresponding
>>>>> repository branch to find its associated “gradle-wrapper.jar”, it is
>>>>> basically free in term of maintenance. Then I agree with Jacques
>>>>> option.
>>>>>
>>>>> Sorry for the confusion.
>>>>>
>>>>> --
>>>>> Mathieu Lirzin
>>>>> GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37
>>>

[Mathieu Lirzin]

Hello Nicolas,

Nicolas Malin <[hidden email]> writes:

> I would had tend to be agree with you with the first step done.
>
> At this time, only [curl,wget] is necessary, and download realease at
> the fisrt gradlew command call.

‘grep’ and ‘whereis’ are currently needed in a Unix environment too. ;-)

> We take the jar, put it directly at the good place and continue the job.

Unfortunately this is not that simple. Here are two important
requirements that the current version of “init-gradle-wrapper.sh” is not
handling:

    1) The integrity of downloaded jar must be checked (via
       checksum). This is important both for reliability and security
       reasons. If we don't do that our users are vulnerable to DNS
       spoofing.

    2) The downloaded jar needs to be automatically upgraded. Currently
       when committers are upgrading Gradle (like I just did in revision
       1862326) the users must delete their “gradle-wrapper.jar”
       manually to not keep using their local outdated version which
       might be incompatible.

In fact the more I think about the problem we are trying to solve, the
more I am convinced we should not try to write our own additional layer
of build system which if done seriously will lead to complex code.

Gradle wrapper is suppposed to be a convenience both for maintainers and
users to facilitate the use of a single and consistent version of
Gradle. IMO with the requirement of not distributing
“gradle-wrapper.jar” in our releases this is becoming a burden for
maintainers and an unreliable solution for users.

As a consequence, I would recommend keeping the wrapper jar in our VCS
repository, and simply remove it along side the “gradlew” scripts from
our release archives. We would then provide preliminary build
instructions stating the required version of Gradle and let users choose
how they want to install it [1].

[1] https://docs.gradle.org/current/userguide/installation.html

--
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37

[Jacques Le Roux]

Hi Mathieu, Nicolas,

@Mathieu, I disagree. IMO this is part of the release process and can be handled by the RM (Release Manager).

When releasing, all the RM has to do is to verify the version in the wrapper is the right one. It it's not, s/he must upgrade Gradle locally and save
the wrapper files in tools.
This saving must be done just after the upgrade. Else the local wrapper files will be overwritten by the ones in tools at the next use of the gradlew
script (yes, I got caught by that :/).

There should be trivial side effect in gradlew scripts due to the Gradle upgrade. Currently only the test on wrapper presence in
init-gradle-wrapper.sh and the ASL2 license header are removed.
It's only one block, easy to put back from repo. This will be documented for the RM as mentioned in OFBIZ-10145.

I'm not the release manager but all committers can do the same for the trunk. And that's what I did.

After applying the last waiting gradlew.bat.patch in OFBIZ-10145 I was able to follow this way.

BTW, I was mislead by Gradle versions names in URL (sometimes?) inconsistent. You have

    https://github.com/gradle/gradle/tree/v5.5.0/gradle/wrapper (5.5.0) (used as RELEASE value in init-gradle-wrapper.sh)

but

    https://services.gradle.org/distributions/gradle-5.5-bin.zip (5.5)

So to update Gradle it's

    |gradlew wrapper --gradle-version=5.5 --distribution-type=all|

not

    gradlew wrapper --gradle-version 5.5.0|--distribution-type=all|

All is better for IDE, default is bin[1].

@Mathieu, I found a trivial issue in your last commits and fixed it: repeated "-Xms64m" in gradlew scripts.

@Nicolas, the version of init-gradle-wrapper.ps1 you committed used HTTPS. I have changed that to HTTP,  not everyone has an HTTPS access to the tools
repo.

Anyway, using the last command worked for me on Windows.

I committed the trunk and tools changes.

[1] https://mrhaki.blogspot.com/2016/09/gradle-goodness-specify-wrapper-version.html

Jacques


Le 30/06/2019 à 17:31, Mathieu Lirzin a écrit :

> Hello Nicolas,
>
> Nicolas Malin <[hidden email]> writes:
>
>> I would had tend to be agree with you with the first step done.
>>
>> At this time, only [curl,wget] is necessary, and download realease at
>> the fisrt gradlew command call.
> ‘grep’ and ‘whereis’ are currently needed in a Unix environment too. ;-)
>
>> We take the jar, put it directly at the good place and continue the job.
> Unfortunately this is not that simple. Here are two important
> requirements that the current version of “init-gradle-wrapper.sh” is not
> handling:
>
>      1) The integrity of downloaded jar must be checked (via
>         checksum). This is important both for reliability and security
>         reasons. If we don't do that our users are vulnerable to DNS
>         spoofing.
>
>      2) The downloaded jar needs to be automatically upgraded. Currently
>         when committers are upgrading Gradle (like I just did in revision
>         1862326) the users must delete their “gradle-wrapper.jar”
>         manually to not keep using their local outdated version which
>         might be incompatible.
>
> In fact the more I think about the problem we are trying to solve, the
> more I am convinced we should not try to write our own additional layer
> of build system which if done seriously will lead to complex code.
>
> Gradle wrapper is suppposed to be a convenience both for maintainers and
> users to facilitate the use of a single and consistent version of
> Gradle. IMO with the requirement of not distributing
> “gradle-wrapper.jar” in our releases this is becoming a burden for
> maintainers and an unreliable solution for users.
>
> As a consequence, I would recommend keeping the wrapper jar in our VCS
> repository, and simply remove it along side the “gradlew” scripts from
> our release archives. We would then provide preliminary build
> instructions stating the required version of Gradle and let users choose
> how they want to install it [1].
>
> [1] https://docs.gradle.org/current/userguide/installation.html
>

[Jacopo Cappellato-3]

On Sun, Jun 30, 2019 at 5:31 PM Mathieu Lirzin <[hidden email]>
wrote:

> [...]
> As a consequence, I would recommend keeping the wrapper jar in our VCS
> repository, and simply remove it along side the “gradlew” scripts from
> our release archives. We would then provide preliminary build
> instructions stating the required version of Gradle and let users choose
> how they want to install it [1].
>
> [1] https://docs.gradle.org/current/userguide/installation.html


I tend to agree with Mathieu.
The above approach would also cover another concern I had with some of the
proposals: since release files can be downloaded by a potentially large
number of users, they must be distributed thru the mirror network [*];
distributing the wrapper jar from our svn, git or website would not scale
up well and could cause excessive load on these server.

Jacopo

[*] https://www.apache.org/dev/release-distribution

>
>

[Jacques Le Roux]

Le 01/07/2019 à 06:53, Jacopo Cappellato a écrit :

> On Sun, Jun 30, 2019 at 5:31 PM Mathieu Lirzin <[hidden email]>
> wrote:
>
>> [...]
>> As a consequence, I would recommend keeping the wrapper jar in our VCS
>> repository, and simply remove it along side the “gradlew” scripts from
>> our release archives. We would then provide preliminary build
>> instructions stating the required version of Gradle and let users choose
>> how they want to install it [1].
>>
>> [1] https://docs.gradle.org/current/userguide/installation.html
>
> I tend to agree with Mathieu.
> The above approach would also cover another concern I had with some of the
> proposals: since release files can be downloaded by a potentially large
> number of users, they must be distributed thru the mirror network [*];
> distributing the wrapper jar from our svn, git or website would not scale
> up well and could cause excessive load on these server.
>
> Jacopo
>
> [*] https://www.apache.org/dev/release-distribution

After working on a Buildbot solution, I suggested this way because I thought it would be easy for our users, and maybe our own tranquillity.

I did not think about scaling. We speak about 55 Kb, the connexions would be very short, so not much at the same time.

It started from this Mark Thomas's remark in LEGAL-288:

    <<The JARs are small and will be downloaded as part of the source tree so I don't think Infra would have any objections.>>

Though with my last proposition, to allow imperceptible Gradle upgrade, this download would be for each call to OFBiz gradlew anywhere. That's indeed
something to consider.

Nicolas suggested to get the wrapper from Git. That can be done, but needs a solution when it disappears there. And it's maybe not fair to them.

I agree, the easiest way for us is certainly to let users follow the standard Gradle installation process as they do for Java.

I'm not against that, just another step that we tried to avoid to our users and their customers.

For working copies checked out (including Buildbot and demos) we can keep the wrapper where it is.

Following LEGAL-288, that's what I initially described in OFBIZ-10145. It was a nice tour :)

Can we conclude?

Jacques

[Swapnil M Mane]

Thanks, everyone for sharing your kind thoughts.
I am also inclined towards allowing users to follow the standard Gradle
installation process.

We tried our best to make the user's life easier by avoiding their
intervention in downloading respective Gradle (resulting the changes in
gradlew and gradlew.bat files and writing init-gradle-wrapper script),
but, it seems this will brings various complexities and maintenance issues
to us like
-- Upgrading the Gradle version
-- Distribution of the wrapper jar

So IMO, let's keep things simple and let the users follow standard Gradle
installation.


Best Regards,
Swapnil M Mane,
ofbiz.apache.org



On Mon, Jul 1, 2019 at 1:52 PM Jacques Le Roux <[hidden email]>
wrote:

> Le 01/07/2019 à 06:53, Jacopo Cappellato a écrit :
> > On Sun, Jun 30, 2019 at 5:31 PM Mathieu Lirzin <
> [hidden email]>
> > wrote:
> >
> >> [...]
> >> As a consequence, I would recommend keeping the wrapper jar in our VCS
> >> repository, and simply remove it along side the “gradlew” scripts from
> >> our release archives. We would then provide preliminary build
> >> instructions stating the required version of Gradle and let users choose
> >> how they want to install it [1].
> >>
> >> [1] https://docs.gradle.org/current/userguide/installation.html
> >
> > I tend to agree with Mathieu.
> > The above approach would also cover another concern I had with some of
> the
> > proposals: since release files can be downloaded by a potentially large
> > number of users, they must be distributed thru the mirror network [*];
> > distributing the wrapper jar from our svn, git or website would not scale
> > up well and could cause excessive load on these server.
> >
> > Jacopo
> >
> > [*] https://www.apache.org/dev/release-distribution
>
> After working on a Buildbot solution, I suggested this way because I
> thought it would be easy for our users, and maybe our own tranquillity.
>
> I did not think about scaling. We speak about 55 Kb, the connexions would
> be very short, so not much at the same time.
>
> It started from this Mark Thomas's remark in LEGAL-288:
>
>     <<The JARs are small and will be downloaded as part of the source tree
> so I don't think Infra would have any objections.>>
>
> Though with my last proposition, to allow imperceptible Gradle upgrade,
> this download would be for each call to OFBiz gradlew anywhere. That's
> indeed
> something to consider.
>
> Nicolas suggested to get the wrapper from Git. That can be done, but needs
> a solution when it disappears there. And it's maybe not fair to them.
>
> I agree, the easiest way for us is certainly to let users follow the
> standard Gradle installation process as they do for Java.
>
> I'm not against that, just another step that we tried to avoid to our
> users and their customers.
>
> For working copies checked out (including Buildbot and demos) we can keep
> the wrapper where it is.
>
> Following LEGAL-288, that's what I initially described in OFBIZ-10145. It
> was a nice tour :)
>
> Can we conclude?
>
> Jacques
>
>
>

[Jacques Le Roux]

A last thing that you pointed to in OFBIZ-10145 Swapnil.

Security issue with a Gradle version used in an already released package.

Like with other security issues we have no other way than letting the user upgrade OFBiz.

So we don't need to "allow imperceptible Gradle upgrade".

Then the connexion to get the wrapper is only once during the installation, the scaling issue is peanuts.

I trust we all agree to keep the wrapper in the trunk. It's not an issue there. We can revert the changes we did in trunk.

It's only a RM issue. It's when packaging that we are tying OFBiz code with a Gradle version.

It's only then that we need to modify the gradlew script, to call an init-gradle-wrapper script if the wrapper is missing.

What do you think folks? Notably you Jacopo, with your RM hat on?

Jacques


Le 01/07/2019 à 13:36, Swapnil M Mane a écrit :

> Thanks, everyone for sharing your kind thoughts.
> I am also inclined towards allowing users to follow the standard Gradle
> installation process.
>
> We tried our best to make the user's life easier by avoiding their
> intervention in downloading respective Gradle (resulting the changes in
> gradlew and gradlew.bat files and writing init-gradle-wrapper script),
> but, it seems this will brings various complexities and maintenance issues
> to us like
> -- Upgrading the Gradle version
> -- Distribution of the wrapper jar
>
> So IMO, let's keep things simple and let the users follow standard Gradle
> installation.
>
>
> Best Regards,
> Swapnil M Mane,
> ofbiz.apache.org
>
>
>
> On Mon, Jul 1, 2019 at 1:52 PM Jacques Le Roux <[hidden email]>
> wrote:
>
>> Le 01/07/2019 à 06:53, Jacopo Cappellato a écrit :
>>> On Sun, Jun 30, 2019 at 5:31 PM Mathieu Lirzin <
>> [hidden email]>
>>> wrote:
>>>
>>>> [...]
>>>> As a consequence, I would recommend keeping the wrapper jar in our VCS
>>>> repository, and simply remove it along side the “gradlew” scripts from
>>>> our release archives. We would then provide preliminary build
>>>> instructions stating the required version of Gradle and let users choose
>>>> how they want to install it [1].
>>>>
>>>> [1] https://docs.gradle.org/current/userguide/installation.html
>>> I tend to agree with Mathieu.
>>> The above approach would also cover another concern I had with some of
>> the
>>> proposals: since release files can be downloaded by a potentially large
>>> number of users, they must be distributed thru the mirror network [*];
>>> distributing the wrapper jar from our svn, git or website would not scale
>>> up well and could cause excessive load on these server.
>>>
>>> Jacopo
>>>
>>> [*] https://www.apache.org/dev/release-distribution
>> After working on a Buildbot solution, I suggested this way because I
>> thought it would be easy for our users, and maybe our own tranquillity.
>>
>> I did not think about scaling. We speak about 55 Kb, the connexions would
>> be very short, so not much at the same time.
>>
>> It started from this Mark Thomas's remark in LEGAL-288:
>>
>>      <<The JARs are small and will be downloaded as part of the source tree
>> so I don't think Infra would have any objections.>>
>>
>> Though with my last proposition, to allow imperceptible Gradle upgrade,
>> this download would be for each call to OFBiz gradlew anywhere. That's
>> indeed
>> something to consider.
>>
>> Nicolas suggested to get the wrapper from Git. That can be done, but needs
>> a solution when it disappears there. And it's maybe not fair to them.
>>
>> I agree, the easiest way for us is certainly to let users follow the
>> standard Gradle installation process as they do for Java.
>>
>> I'm not against that, just another step that we tried to avoid to our
>> users and their customers.
>>
>> For working copies checked out (including Buildbot and demos) we can keep
>> the wrapper where it is.
>>
>> Following LEGAL-288, that's what I initially described in OFBIZ-10145. It
>> was a nice tour :)
>>
>> Can we conclude?
>>
>> Jacques
>>
>>
>>

[Jacques Le Roux]

Le 02/07/2019 à 07:13, Jacques Le Roux a écrit :
> Then the connexion to get the wrapper is only once during the installation, the scaling issue is peanuts.

If ever we worry about the load on the ASF svn (or git) server (really we should not), why not use Gradle Github as suggested Nicolas?

My concern was that sometimes the wrapper disappears there. But that should be only in a far future for old, no longer supported, OFBiz releases.

If we do so we could also use https://services.gradle.org/distributions to verify the wrapper checksum.

Sincerely I don't think it's needed, I just mention it as a second argument.

Jacques

[Jacopo Cappellato-3]

On Tue, Jul 2, 2019 at 7:14 AM Jacques Le Roux <[hidden email]>
wrote:

> [...]


> It's only a RM issue. It's when packaging that we are tying OFBiz code
> with a Gradle version.
>
> It's only then that we need to modify the gradlew script, to call an
> init-gradle-wrapper script if the wrapper is missing.
>
> What do you think folks? Notably you Jacopo, with your RM hat on?


I think that the direction we are heading is to not include it in the
releases and add instructions in the release's README file to tell the user
how to download it.
So we could keep the wrappers (if we like) in the trunk and/or release
branches and remove them when we package/publish a new release.

Jacopo