Repository Management For Third-Party Plugins

[Pritam Kute]

Hello Devs,

Recently, we have pushed integration of the First Data payment gateway to
OOTB. When we were doing that, we got one question about repository
management for third-party applications. We have one ticket [1] open in
JIRA about moving third party code from framework to separate repositories
to make framework lightweight.

Now the question is how we are going to manage the repository structure for
third party integration codes. If we talk about payment gateways, we have
around 12 different integrations available in the framework component. If
we move them to plugins as a separate plugin, it will increase the number
of plugins in the plugin repository. I am personally not in favor of that.
We should have some type of repository categorization there. One
categorization I would like to suggest is to have separate repository all
together for third party integrations and to have a Gradle task to fetch
only required third party integration inside plugins directory as like we
have done for plugins repository itself:
apache -
    -- ofbiz-framework
    -- ofbiz-plugins
    -- *ofbiz-thrid-party*
    -- ofbiz-site
    -- ofbiz-tools

I would like to invite thoughts on this so that we can proceed with the
migration of third party code out of the framework.

[1] https://issues.apache.org/jira/browse/OFBIZ-7415

Kind Regards,
--
Pritam Kute

[Mridul Pathak-4]

I recall having at length discussions on this topic in the mentioned ticket back then. I too was a bit concerned about having too many plugins for similar type of code like payment integration or shipping integrations. But now when I think of it again, the core idea behind plugin architecture is to be able to build any number of plugins on an existing product/framework. So I think it’s fine having any number of individual plugins for these third party integrations, in fact many of the plugins today are third party integrations. We shouldn’t be needing another repository for this.

Though I would definitely suggest that we should review each of these third party integrations and only move those which are valid implementations today. There could be certain integrations which might be deprecated or defunct now due to lack of maintenance and needs to be updated or rewritten for their latest versions. It could be a good idea to create place holder Jira issues for reimplementation for contributors to pick.

Thanks.
--
Mridul Pathak

> On 25-Jul-2020, at 11:59 AM, Pritam Kute <[hidden email]> wrote:
>
> Hello Devs,
>
> Recently, we have pushed integration of the First Data payment gateway to
> OOTB. When we were doing that, we got one question about repository
> management for third-party applications. We have one ticket [1] open in
> JIRA about moving third party code from framework to separate repositories
> to make framework lightweight.
>
> Now the question is how we are going to manage the repository structure for
> third party integration codes. If we talk about payment gateways, we have
> around 12 different integrations available in the framework component. If
> we move them to plugins as a separate plugin, it will increase the number
> of plugins in the plugin repository. I am personally not in favor of that.
> We should have some type of repository categorization there. One
> categorization I would like to suggest is to have separate repository all
> together for third party integrations and to have a Gradle task to fetch
> only required third party integration inside plugins directory as like we
> have done for plugins repository itself:
> apache -
>    -- ofbiz-framework
>    -- ofbiz-plugins
>    -- *ofbiz-thrid-party*
>    -- ofbiz-site
>    -- ofbiz-tools
>
> I would like to invite thoughts on this so that we can proceed with the
> migration of third party code out of the framework.
>
> [1] https://issues.apache.org/jira/browse/OFBIZ-7415
>
> Kind Regards,
> --
> Pritam Kute

[Pawan Verma]

Hi Pritam,

I am also more inclined to not create a different repository for
third-party-plugins. As we can have any number of plugins in OFBiz and
Users can set up required plugins as per their need.

I would suggest creating separate plugins for each third-party integration
like ups, firstdatapaymentgateway, etc that way if the user wants only a
specific gateway or plugins can have only that one.

I suggest two approaches:
#1: Create a separate plugin for each integration and put them into the
plugins directory.

#2: Club all the related plugins into one directory and can add that parent
directory in base/config/component-load.xml. The benefit of this approach
is we can have all payment or other gateways in one place and also don't
need to create a separate repository for the third-party-plugins.
Example for shipping gateways:

   1. Club all shipping gateways into the plugins/shipping-gateway directory
   2. Add this shipping-gateway into base/config/component-load.xml
   as <load-components parent-directory="plugins/shipping-gateway"/>

If we follow the #2 approach we may need to improve our plugin setup Gradle
target i.e. pullPluginSource.

Please let me know your thoughts and I can provide more detail if needed.

Best
Pawan

[Mridul Pathak-4]

Hi Pawan,

Though few years ago I was advocate of the approach #2 that you have suggested here, now I think that #1 is the best approach. In real time implementations the idea of plugins is to be able to use only what you need. Like for payment integrations you selectively use the gateways, mostly one. So it should be as easy to drop the plugin for your preferred gateway and use it, you shouldn’t need to unnecessarily have to code of all other payment integrations that you don’t need. Also, from maintenance perspective even though some of these third party integrations could be categorised but still each of them have their own API specification, configurations and they need to be maintained separately. So, my vote would be to #1 that is to create separate plugins for each third party integration.

Thanks.
--
Mridul Pathak

> On 30-Jul-2020, at 8:34 PM, Pawan Verma <[hidden email]> wrote:
>
> Hi Pritam,
>
> I am also more inclined to not create a different repository for
> third-party-plugins. As we can have any number of plugins in OFBiz and
> Users can set up required plugins as per their need.
>
> I would suggest creating separate plugins for each third-party integration
> like ups, firstdatapaymentgateway, etc that way if the user wants only a
> specific gateway or plugins can have only that one.
>
> I suggest two approaches:
> #1: Create a separate plugin for each integration and put them into the
> plugins directory.
>
> #2: Club all the related plugins into one directory and can add that parent
> directory in base/config/component-load.xml. The benefit of this approach
> is we can have all payment or other gateways in one place and also don't
> need to create a separate repository for the third-party-plugins.
> Example for shipping gateways:
>
>   1. Club all shipping gateways into the plugins/shipping-gateway directory
>   2. Add this shipping-gateway into base/config/component-load.xml
>   as <load-components parent-directory="plugins/shipping-gateway"/>
>
> If we follow the #2 approach we may need to improve our plugin setup Gradle
> target i.e. pullPluginSource.
>
> Please let me know your thoughts and I can provide more detail if needed.
>
> Best
> Pawan

[Jacques Le Roux]

Hi,

Like Mridul, I also suggested  the 2nd approach. I eventually tend to agree with Mridul. Having clearly separated packages, even with a bit of
redundancy, simplifies and consolidate things.

Jacques

Le 30/07/2020 à 17:37, Mridul Pathak a écrit :

> Hi Pawan,
>
> Though few years ago I was advocate of the approach #2 that you have suggested here, now I think that #1 is the best approach. In real time implementations the idea of plugins is to be able to use only what you need. Like for payment integrations you selectively use the gateways, mostly one. So it should be as easy to drop the plugin for your preferred gateway and use it, you shouldn’t need to unnecessarily have to code of all other payment integrations that you don’t need. Also, from maintenance perspective even though some of these third party integrations could be categorised but still each of them have their own API specification, configurations and they need to be maintained separately. So, my vote would be to #1 that is to create separate plugins for each third party integration.
>
> Thanks.
> --
> Mridul Pathak
>
>
>> On 30-Jul-2020, at 8:34 PM, Pawan Verma <[hidden email]> wrote:
>>
>> Hi Pritam,
>>
>> I am also more inclined to not create a different repository for
>> third-party-plugins. As we can have any number of plugins in OFBiz and
>> Users can set up required plugins as per their need.
>>
>> I would suggest creating separate plugins for each third-party integration
>> like ups, firstdatapaymentgateway, etc that way if the user wants only a
>> specific gateway or plugins can have only that one.
>>
>> I suggest two approaches:
>> #1: Create a separate plugin for each integration and put them into the
>> plugins directory.
>>
>> #2: Club all the related plugins into one directory and can add that parent
>> directory in base/config/component-load.xml. The benefit of this approach
>> is we can have all payment or other gateways in one place and also don't
>> need to create a separate repository for the third-party-plugins.
>> Example for shipping gateways:
>>
>>    1. Club all shipping gateways into the plugins/shipping-gateway directory
>>    2. Add this shipping-gateway into base/config/component-load.xml
>>    as <load-components parent-directory="plugins/shipping-gateway"/>
>>
>> If we follow the #2 approach we may need to improve our plugin setup Gradle
>> target i.e. pullPluginSource.
>>
>> Please let me know your thoughts and I can provide more detail if needed.
>>
>> Best
>> Pawan

[Pawan Verma]

Thanks Mridul and Jacques for the valuable feedback.

I also agree with your proposition. My vote is also for approach #1.

Best
Pawan

On Thu, Jul 30, 2020, 9:32 PM Jacques Le Roux <[hidden email]>
wrote:

> Hi,
>
> Like Mridul, I also suggested  the 2nd approach. I eventually tend to
> agree with Mridul. Having clearly separated packages, even with a bit of
> redundancy, simplifies and consolidate things.
>
> Jacques
>
> Le 30/07/2020 à 17:37, Mridul Pathak a écrit :
> > Hi Pawan,
> >
> > Though few years ago I was advocate of the approach #2 that you have
> suggested here, now I think that #1 is the best approach. In real time
> implementations the idea of plugins is to be able to use only what you
> need. Like for payment integrations you selectively use the gateways,
> mostly one. So it should be as easy to drop the plugin for your preferred
> gateway and use it, you shouldn’t need to unnecessarily have to code of all
> other payment integrations that you don’t need. Also, from maintenance
> perspective even though some of these third party integrations could be
> categorised but still each of them have their own API specification,
> configurations and they need to be maintained separately. So, my vote would
> be to #1 that is to create separate plugins for each third party
> integration.
> >
> > Thanks.
> > --
> > Mridul Pathak
> >
> >
> >> On 30-Jul-2020, at 8:34 PM, Pawan Verma <[hidden email]>
> wrote:
> >>
> >> Hi Pritam,
> >>
> >> I am also more inclined to not create a different repository for
> >> third-party-plugins. As we can have any number of plugins in OFBiz and
> >> Users can set up required plugins as per their need.
> >>
> >> I would suggest creating separate plugins for each third-party
> integration
> >> like ups, firstdatapaymentgateway, etc that way if the user wants only a
> >> specific gateway or plugins can have only that one.
> >>
> >> I suggest two approaches:
> >> #1: Create a separate plugin for each integration and put them into the
> >> plugins directory.
> >>
> >> #2: Club all the related plugins into one directory and can add that
> parent
> >> directory in base/config/component-load.xml. The benefit of this
> approach
> >> is we can have all payment or other gateways in one place and also don't
> >> need to create a separate repository for the third-party-plugins.
> >> Example for shipping gateways:
> >>
> >>    1. Club all shipping gateways into the plugins/shipping-gateway
> directory
> >>    2. Add this shipping-gateway into base/config/component-load.xml
> >>    as <load-components parent-directory="plugins/shipping-gateway"/>
> >>
> >> If we follow the #2 approach we may need to improve our plugin setup
> Gradle
> >> target i.e. pullPluginSource.
> >>
> >> Please let me know your thoughts and I can provide more detail if
> needed.
> >>
> >> Best
> >> Pawan
>
>

[Pritam Kute]

Thanks to all for putting your thoughts together. We will start working on
moving third party code out of the framework soon.

As suggested by Mridul, we will take one integration at a time and validate
if it is working and up to date with current scenarios and migrate only
those integrations which are valid implementations today.

Kind Regards,
--
Pritam Kute


On Thu, Jul 30, 2020 at 10:23 PM Pawan Verma <[hidden email]>
wrote:

> Thanks Mridul and Jacques for the valuable feedback.
>
> I also agree with your proposition. My vote is also for approach #1.
>
> Best
> Pawan
>
> On Thu, Jul 30, 2020, 9:32 PM Jacques Le Roux <
> [hidden email]>
> wrote:
>
> > Hi,
> >
> > Like Mridul, I also suggested  the 2nd approach. I eventually tend to
> > agree with Mridul. Having clearly separated packages, even with a bit of
> > redundancy, simplifies and consolidate things.
> >
> > Jacques
> >
> > Le 30/07/2020 à 17:37, Mridul Pathak a écrit :
> > > Hi Pawan,
> > >
> > > Though few years ago I was advocate of the approach #2 that you have
> > suggested here, now I think that #1 is the best approach. In real time
> > implementations the idea of plugins is to be able to use only what you
> > need. Like for payment integrations you selectively use the gateways,
> > mostly one. So it should be as easy to drop the plugin for your preferred
> > gateway and use it, you shouldn’t need to unnecessarily have to code of
> all
> > other payment integrations that you don’t need. Also, from maintenance
> > perspective even though some of these third party integrations could be
> > categorised but still each of them have their own API specification,
> > configurations and they need to be maintained separately. So, my vote
> would
> > be to #1 that is to create separate plugins for each third party
> > integration.
> > >
> > > Thanks.
> > > --
> > > Mridul Pathak
> > >
> > >
> > >> On 30-Jul-2020, at 8:34 PM, Pawan Verma <
> [hidden email]>
> > wrote:
> > >>
> > >> Hi Pritam,
> > >>
> > >> I am also more inclined to not create a different repository for
> > >> third-party-plugins. As we can have any number of plugins in OFBiz and
> > >> Users can set up required plugins as per their need.
> > >>
> > >> I would suggest creating separate plugins for each third-party
> > integration
> > >> like ups, firstdatapaymentgateway, etc that way if the user wants
> only a
> > >> specific gateway or plugins can have only that one.
> > >>
> > >> I suggest two approaches:
> > >> #1: Create a separate plugin for each integration and put them into
> the
> > >> plugins directory.
> > >>
> > >> #2: Club all the related plugins into one directory and can add that
> > parent
> > >> directory in base/config/component-load.xml. The benefit of this
> > approach
> > >> is we can have all payment or other gateways in one place and also
> don't
> > >> need to create a separate repository for the third-party-plugins.
> > >> Example for shipping gateways:
> > >>
> > >>    1. Club all shipping gateways into the plugins/shipping-gateway
> > directory
> > >>    2. Add this shipping-gateway into base/config/component-load.xml
> > >>    as <load-components parent-directory="plugins/shipping-gateway"/>
> > >>
> > >> If we follow the #2 approach we may need to improve our plugin setup
> > Gradle
> > >> target i.e. pullPluginSource.
> > >>
> > >> Please let me know your thoughts and I can provide more detail if
> > needed.
> > >>
> > >> Best
> > >> Pawan
> >
> >
>

[Nicolas Malin-2]

Hi,

I have the feeling that the first approch is a step to move on second.

Create a tree structure for plugins would be necessary and implement it
the resolve installed plugins without framework change for each new sub
directory seems a mandatory way to keep it simple.

Nicolas

On 30/07/2020 18:53, Pawan Verma wrote:

> Thanks Mridul and Jacques for the valuable feedback.
>
> I also agree with your proposition. My vote is also for approach #1.
>
> Best
> Pawan
>
> On Thu, Jul 30, 2020, 9:32 PM Jacques Le Roux <[hidden email]>
> wrote:
>
>> Hi,
>>
>> Like Mridul, I also suggested  the 2nd approach. I eventually tend to
>> agree with Mridul. Having clearly separated packages, even with a bit of
>> redundancy, simplifies and consolidate things.
>>
>> Jacques
>>
>> Le 30/07/2020 à 17:37, Mridul Pathak a écrit :
>>> Hi Pawan,
>>>
>>> Though few years ago I was advocate of the approach #2 that you have
>> suggested here, now I think that #1 is the best approach. In real time
>> implementations the idea of plugins is to be able to use only what you
>> need. Like for payment integrations you selectively use the gateways,
>> mostly one. So it should be as easy to drop the plugin for your preferred
>> gateway and use it, you shouldn’t need to unnecessarily have to code of all
>> other payment integrations that you don’t need. Also, from maintenance
>> perspective even though some of these third party integrations could be
>> categorised but still each of them have their own API specification,
>> configurations and they need to be maintained separately. So, my vote would
>> be to #1 that is to create separate plugins for each third party
>> integration.
>>> Thanks.
>>> --
>>> Mridul Pathak
>>>
>>>
>>>> On 30-Jul-2020, at 8:34 PM, Pawan Verma <[hidden email]>
>> wrote:
>>>> Hi Pritam,
>>>>
>>>> I am also more inclined to not create a different repository for
>>>> third-party-plugins. As we can have any number of plugins in OFBiz and
>>>> Users can set up required plugins as per their need.
>>>>
>>>> I would suggest creating separate plugins for each third-party
>> integration
>>>> like ups, firstdatapaymentgateway, etc that way if the user wants only a
>>>> specific gateway or plugins can have only that one.
>>>>
>>>> I suggest two approaches:
>>>> #1: Create a separate plugin for each integration and put them into the
>>>> plugins directory.
>>>>
>>>> #2: Club all the related plugins into one directory and can add that
>> parent
>>>> directory in base/config/component-load.xml. The benefit of this
>> approach
>>>> is we can have all payment or other gateways in one place and also don't
>>>> need to create a separate repository for the third-party-plugins.
>>>> Example for shipping gateways:
>>>>
>>>>    1. Club all shipping gateways into the plugins/shipping-gateway
>> directory
>>>>    2. Add this shipping-gateway into base/config/component-load.xml
>>>>    as <load-components parent-directory="plugins/shipping-gateway"/>
>>>>
>>>> If we follow the #2 approach we may need to improve our plugin setup
>> Gradle
>>>> target i.e. pullPluginSource.
>>>>
>>>> Please let me know your thoughts and I can provide more detail if
>> needed.
>>>> Best
>>>> Pawan
>>