Re: [DISCUSSION] Plugins: svn:external or Gradle Task?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
27 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSSION] Plugins: svn:external or Gradle Task?

Michael Brohl-3
Hi Taher,

that would be the valid process for a contributor. For me as a
committer, I'd like to directly commit a change to the repository
(either after applying a patch or after a direct change).

I'm currently not sure if I oversee all possibilities we have out of the
box. Jacques mentioned that there is no problem and I might have to
check myself how this works out.

So maybe we have no issue. I'll need time to test the development
workflows with the two repositories first before I take more of your
time to discuss ;-)

Cheers,

Michael


Am 13.02.17 um 11:01 schrieb Taher Alkhateeb:

> Hi Michael. If I understand your point correctly regarding ease of
> contribution, would it be a problem to just go to the plugin directory and
> issue a command like svn diff > plugin-fix.patch?
>
> On Mon, Feb 13, 2017 at 12:54 PM, Michael Brohl <[hidden email]>
> wrote:
>
>> Hi Taher,
>>
>> inline...
>>
>> Am 13.02.17 um 06:33 schrieb Taher Alkhateeb:
>>
>>> Hi Michael, All
>>>
>>> All good points, thank you for your thoughts. So a solution that I
>>> proposed
>>> in jira to easily manage and maintain our official plugins is to have a
>>> gradle task named for example pullPluginSourceAll which pulls all the
>>> plugins in one shot for testing and development purposes. Maybe we can
>>> also
>>> add logic in this task so that it updates already downloaded plugins.
>>>
>> That's a good approach if you want to test plugins/integration within the
>> framework without the need to work on plugin sources under source control,
>> if I understand it right. This should be the default case.
>>
>> Additionally, I suggest to provide a task which creates the svn:externals
>> file for the case where you want the plugin sources in your project and
>> have them under source control.
>> Or, if we don't want to have such a task in the codebase, provide some
>> documentation on how to achieve this manually.
>> Maybe there are other good solutions as well.
>>
>> Else I cannot think of an easy process for a plugin developer to create
>> patches or make commits against the repository without hassle.
>>
>>> I'm sure there are other good ideas as well. My point of emphasis is to
>>> Simply keep the two projects clearly separated, and to avoid relying
>>> directly on tools not embedded within the framework like subversion.
>>>
>> I completely agree on this point and this should be the default. I just
>> think that we should provide some (non default) alternatives or at least
>> how-to's to make it easy for everyone to contribute.
>> We should think about different types of users and contributors.
>>
>> Regards,
>>
>> Michael
>>
>>
>>> On Feb 13, 2017 12:08 AM, "Michael Brohl" <[hidden email]>
>>> wrote:
>>>
>>> Hi,
>>>> first of all: great work and many thanks to Taher, Jacques and Deepak for
>>>> their efforts on the repository separation this weekend.
>>>>
>>>> I think we should provide processes and support for the following cases:
>>>>
>>>> 1. separated development of the framework and core applications without
>>>> forced embedding of the plugins
>>>>
>>>> 2. separated development of the plugins with the support to embed the
>>>> framework part in the development environment
>>>>
>>>> 3. combined development of both framework and plugins.
>>>>
>>>> I think we'll need convenience tools/processes/support for these
>>>> combinations because we should make contributions for and use of
>>>> framework
>>>> + plugins as easy as possible without forcing the entanglement of the
>>>> different projects.
>>>>
>>>> As far as I understand forced use of svn:externals is contra productive
>>>> because it prevents the separation. We should instead describe the
>>>> mechanism and provide a set of svn:externals files for all users who want
>>>> to use it. Maybe also a gradle task which generates the needed file(s)
>>>> for
>>>> convenience.
>>>>
>>>> I think we cannot only rely on the pull plugin mechanism because combined
>>>> development needs to have framework and plugins in the IDE under version
>>>> control.
>>>>
>>>> Users and developers should have a choice (and low hurdles to
>>>> contribute),
>>>> on the other hand we should keep it lean and simple per default.
>>>>
>>>> Not sure if I could make my viewpoint clear enough, I'm currently too
>>>> tired to concentrate on better words :-)
>>>>
>>>> Best regards,
>>>>
>>>> Michael
>>>>
>>>>
>>>> Am 12.02.17 um 20:59 schrieb Taher Alkhateeb:
>>>>
>>>> Hi Sharan,
>>>>> I think this is not the topic under discussion, and since it seems vague
>>>>> to
>>>>> you I will try to clarify:
>>>>>
>>>>> The question now that we split framework and plugins is how to combine
>>>>> them. The discussion so far between myself and Jacques is as follows:
>>>>> - Jacques wants to use svn:externals so that subversion pulls all the
>>>>> plugins into the framework and keep everything as is (testing, builds,
>>>>> etc)
>>>>> the way they were.
>>>>> - My position is that we should use gradle to pull the plugins we need
>>>>> without reliance on Subversion or any other tool and to treat the two
>>>>> projects as two separate projects.
>>>>>
>>>>> Both reasons / concerns from me and Jacques are expressed by reading the
>>>>> history of this thread (that partly exists in another thread for which
>>>>> Deepak contributed his opinion).
>>>>>
>>>>> On Sun, Feb 12, 2017 at 10:49 PM, Sharan Foga <[hidden email]>
>>>>> wrote:
>>>>>
>>>>> Hi All
>>>>>
>>>>>> I'm not exactly completely sure what this discussion is all about. The
>>>>>> title talks about svn and Gradle but questions in the thread seem to
>>>>>> about
>>>>>> something different. If it is as important as you say then we need to
>>>>>> make
>>>>>> sure that everyone understands exactly what this means.
>>>>>>
>>>>>> Are you saying that because of the framework / plugin split then
>>>>>> committers will work only on the framework? If so, then remember that
>>>>>> it
>>>>>> was our contributors and committers that put the 'plugins' into OFBiz
>>>>>> in
>>>>>> the first place which means that people who are interested in
>>>>>> developing
>>>>>> a
>>>>>> particular functionality further can do it without worrying about or
>>>>>> breaking the framework.
>>>>>>
>>>>>>    From my side I know that we also have some OFBiz application
>>>>>> documentation
>>>>>> that is sitting in docbook inside OFBiz I think would be better
>>>>>> implemented
>>>>>> as a plugin. It would be good to be able to work on and test this
>>>>>> separately.
>>>>>>
>>>>>> I apologise if I've misunderstood this thread but as I say - it's not
>>>>>> very
>>>>>> clear what it is all about.
>>>>>>
>>>>>> Thanks
>>>>>> Sharan
>>>>>>
>>>>>> On 2017-02-12 13:55 (+0100), Jacques Le Roux <
>>>>>> [hidden email]>
>>>>>> wrote:
>>>>>>
>>>>>> Thanks Taher,
>>>>>>> I create a clean new thread to have it more prominent.
>>>>>>>
>>>>>>> This is really what needs to be discussed indeed.
>>>>>>>
>>>>>>> Are we sure that's what we want for plugins?
>>>>>>>
>>>>>>> Will they not be disregarded by committers?
>>>>>>>
>>>>>>> Will plugins follow the framework trend?
>>>>>>>
>>>>>>> This is a very important community decision.
>>>>>>> I hope everybody will (try to) understand the consequences and
>>>>>>>
>>>>>>> participate to this discussion, before it will be too late...
>>>>>> Jacques
>>>>>>>
>>>>>>> Le 12/02/2017 � 13:32, Taher Alkhateeb a �crit :
>>>>>>>
>>>>>>> Let's take a look at the big picture for a moment before we decide. We
>>>>>>>> have
>>>>>>> broken OFBiz into essentially two products: framework and plugins.
>>>>>>>
>>>>>>>> Why did we break it up? Why did we go through the trouble of creating
>>>>>>>>
>>>>>>>> two
>>>>>>> different repositories? I hope the answer is something like "because
>>>>>>> it is
>>>>>>> too big" or "because it carried a lot of non-essential functionality
>>>>>>> around"
>>>>>>> So breaking up the project brings the advantage of not having to
>>>>>>> _worry
>>>>>>>
>>>>>>>> about everything_ on each and every commit. These are two projects
>>>>>>>>
>>>>>>>> with two
>>>>>>> release cycles and they don't need to be developed together
>>>>>>> simultaneously.
>>>>>>> People can specialize and focus on different areas.
>>>>>>>
>>>>>>>> The whole idea of breaking the system up and then using gradle's
>>>>>>>>
>>>>>>>> plugin API
>>>>>>> for OFBiz is to remove the coupling between the two projects. If we
>>>>>>>
>>>>>>>> introduce svn externals then all  we achieve is create two
>>>>>>>> repositories
>>>>>>>> only to merge them again. Why! that's pointless!
>>>>>>>>
>>>>>>>> Two products means two buildbot scripts, two releases, two
>>>>>>>> development
>>>>>>>> cycles, two different things!
>>>>>>>>
>>>>>>>> So to achieve true separation and decoupling, I suggest to avoid any
>>>>>>>>
>>>>>>>> hacks
>>>>>>> like svn externals.
>>>>>>>
>>>>>>>> On Sun, Feb 12, 2017 at 1:31 PM, Jacques Le Roux <
>>>>>>>> [hidden email]> wrote:
>>>>>>>>
>>>>>>>> Hi All,
>>>>>>>>
>>>>>>>>> This discussion already happened at OFBIZ-9182, but only between
>>>>>>>>>
>>>>>>>>> Taher,
>>>>>>> Nicolas and I.
>>>>>>>
>>>>>>>> The question is simple, below reflects it, and the subject summarises
>>>>>>>>> it.
>>>>>>> So what are your thoughts and opinions?
>>>>>>>
>>>>>>>> Also when answering I guess we can remove  '(was Re: Proposal to
>>>>>>>>> create a
>>>>>>> separate svn repository for the OFBiz official plugins)' from the
>>>>>>>
>>>>>>>> subject
>>>>>>>>
>>>>>>> (I was just "lazy" here)
>>>>>>>
>>>>>>>> Thanks
>>>>>>>>> Jacques
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Le 12/02/2017 � 11:22, Jacques Le Roux a �crit :
>>>>>>>>>
>>>>>>>>> Sincerely I hardly see the benefit, but I see the disadvantages when
>>>>>>>>> I
>>>>>>>>>
>>>>>>>> remember what happened with R13.07. I mean how and by who will be
>>>>>>>> maintained the OOTB plugins?
>>>>>>>>>> I think this should be more discussed, and maybe voted, here
>>>>>>>>>>
>>>>>>>>>> Jacques
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Le 12/02/2017 � 11:18, Deepak Dixit a �crit :
>>>>>>>>>>
>>>>>>>>>> Hi Jacques,
>>>>>>>>>>
>>>>>>>>>>> We can add gradle task to  pull all plugins from remote. As we are
>>>>>>>>>>> de-coupling plugins from core so I think its good idea to keep
>>>>>>>>>>> them
>>>>>>>>>>> separate. If any committer or developer want he can use gradle
>>>>>>>>>>> task
>>>>>>>>>>>
>>>>>>>>>>> for
>>>>>>>>> the
>>>>>>>> same.
>>>>>>>>>>> Thanks & Regards
>>>>>>>>>>> --
>>>>>>>>>>> Deepak Dixit
>>>>>>>>>>> www.hotwaxsystems.com
>>>>>>>>>>>
>>>>>>>>>>> On Sun, Feb 12, 2017 at 3:43 PM, Jacques Le Roux <
>>>>>>>>>>> [hidden email]> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Yes this is the idea, why should we not? How else committers will
>>>>>>>>>>>
>>>>>>>>>>> easily
>>>>>>>>> maintain the plugins?
>>>>>>>> Jacques
>>>>>>>>>>>>
>>>>>>>>>>>> Le 12/02/2017 � 10:25, Deepak Dixit a �crit :
>>>>>>>>>>>>
>>>>>>>>>>>> Hi Jacques,
>>>>>>>>>>>>
>>>>>>>>>>>> I think if wesvn:external  on trunk, then it will always checkout
>>>>>>>>>>>>> the
>>>>>>>>>>> plugins with trunk
>>>>>>>>>>>>> Thanks & Regards
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Deepak Dixit
>>>>>>>>>>>>> www.hotwaxsystems.com
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Sun, Feb 12, 2017 at 2:52 PM, Deepak Dixit <
>>>>>>>>>>>>> [hidden email]> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>


smime.p7s (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSSION] Plugins: svn:external or Gradle Task?

Jacques Le Roux
Administrator
In reply to this post by Jacques Le Roux
Le 13/02/2017 à 10:56, Jacques Le Roux a écrit :
I have an idea for that: we create a copy of the trunk in a BuildBot special branch, say ofbiz-framework-buildbot

This branch must never be used to commit directly (we can always revert in case)

This branch has <a moz-do-not-send="true" class="moz-txt-link-freetext" href="svn:externals">svn:externals for plugins installed has described previously

In Buildbot, we no longer build the trunk framework and plugins.

But, on a svn commit in ofbiz-framework, or in the ofbiz-plugins, we merge the commit in ofbiz-framework-buildbot and we run the tests and the rest

I have though still to figure out how to trigger a build on a commit external to this branch, and even if it's possible...

Good news, my Buildbot trick works:
https://ci.apache.org/builders/ofbiz-trunk-framework/builds/3

https://ci.apache.org/builders/ofbiz-trunk-plugins/builds/1

I need to update the snapshots page again by adding the new archive. I'm waiting for a Puppet-Github pull request after cleaning https://github.com/apache/infrastructure-puppet/pull/180

Actually the trick is to have a ofbiz-framework-buildbot branch which is only constituted of <a class="moz-txt-link-freetext" href="svn:externals">svn:externals. So no need to worry about updating, merging it, etc.
This branch has a prominent file named "ONLY FOR BUILDBOT TESTS DON'T USE FOR DEVELOPEMENT" to make things clear

We now just need INFRA-13496 to be fixed <img class="remoticon" aria-label=":)" alt=":)" src="http://ofbiz.116.s1.nabble.com/attachment/4702365/0/smile%402x.png" onerror="if (HC.emoticon_resolution_helper) { HC.emoticon_resolution_helper(this); }" height="20" width="20"> (the port 8080 issue again)

Have a nice day
Jacques
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSSION] Plugins: svn:external or Gradle Task?

Jacques Le Roux
Administrator
In reply to this post by Jacques Le Roux
Le 13/02/2017 à 10:56, Jacques Le Roux a écrit :
> So, apart if I missed something, technically it's OK. I believe committers need now to agree about checking out all plugins and maintaining them as
> we did before. This must be documented in our policies.
We also need to better, but as simply as possible, document the change.

We will certainly get more than https://issues.apache.org/jira/browse/OFBIZ-9214

Jacques

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSSION] Plugins: svn:external or Gradle Task?

Jacques Le Roux
Administrator
In reply to this post by Jacques Le Roux
Le 13/02/2017 à 10:56, Jacques Le Roux a écrit :
> So updting  a plugin by hand with svn, or even several plugins at a time with an IDE or a Svn GUI , is not be a problem, nor to commit changes, etc.
> So we don't need Gradle tasks for that, all is still possible.
>
> To summarize, so far I seed no disadvantages with Taher solution, but in Builbot for plugins.
Something I forgot to talk about and, though related to rare cases, is worth to mention for committers. Each time we use pullPluginSource Gradle task,
we create an embedded working copy for the imported component. With the to come pullAllPluginsSource Gradle task we can't checkout  the whole plugins
folder in an embedded working copy. It would be in conflict with what pullPluginSource Gradle task creates. So we have to create as much as  embedded
working copies as components in the plugins folder. So in rare cases were a lot of components are modified we will need to do as much as commits as
component are changed.

Jacques

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSSION] Plugins: svn:external or Gradle Task?

Jacques Le Roux
Administrator
In reply to this post by Jacques Le Roux
Le 14/02/2017 à 08:57, Jacques Le Roux a écrit :
We now just need INFRA-13496 to be fixed <img shrinktofit="true" class="remoticon" aria-label=":)" alt=":)" src="http://ofbiz.116.s1.nabble.com/attachment/4703075/0/part1.E918A25E.91EE3F51%40les7arts.com" onerror="if (HC.emoticon_resolution_helper) { HC.emoticon_resolution_helper(this); }" height="20" width="20"> (the port 8080 issue again)

Have a nice day
Jacques
Here we go again https://issues.apache.org/jira/servicedesk/customer/portal/1/INFRA-13573
Jacques
Reply | Threaded
Open this post in threaded view
|

Test and ports issue (was Re: [DISCUSSION] Plugins: svn:external or Gradle Task?)

Jacques Le Roux
Administrator
Le 08/03/2017 à 19:57, Jacques Le Roux a écrit :
Le 14/02/2017 à 08:57, Jacques Le Roux a écrit :
We now just need INFRA-13496 to be fixed <img shrinktofit="true" class="remoticon" aria-label=":)" alt=":)" src="http://ofbiz.116.s1.nabble.com/attachment/4703125/0/part1.DB158131.1E0DAEB7%40les7arts.com" onerror="if (HC.emoticon_resolution_helper) { HC.emoticon_resolution_helper(this); }" height="20" width="20"> (the port 8080 issue again)

Have a nice day
Jacques
Here we go again https://issues.apache.org/jira/servicedesk/customer/portal/1/INFRA-13573
Jacques
OK this is finally resolved, see (unrelated but with links) OFBIZ-9196

Jacques
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSSION] Plugins: svn:external or Gradle Task?

Jacques Le Roux
Administrator
In reply to this post by Jacques Le Roux
Le 14/02/2017 à 09:08, Jacques Le Roux a écrit :

> Le 13/02/2017 à 10:56, Jacques Le Roux a écrit :
>> So, apart if I missed something, technically it's OK. I believe committers need now to agree about checking out all plugins and maintaining them as
>> we did before. This must be documented in our policies.
> We also need to better, but as simply as possible, document the change.
>
> We will certainly get more than https://issues.apache.org/jira/browse/OFBIZ-9214
>
> Jacques
>
>
I think we have already detailed enough the trunk split with our last entry in the blog https://blogs.apache.org/ofbiz/  which refers to the changes I
made in our "OFBiz Source Repositories" page

Done for me, I don't see what adding a wiki page would add. If you think otherwise please feel free to do so.

Jacques

12