Hi Experts,
I would like to embed OfBiz with a module environment (not OSGi - but sufficiently similar to think OSGi if that helps) and for multiple tenants on one VM. The latter is the the stronger requirement for me. But as far as modularization is concerned, Ofbiz seems to be in good shape: The individual logical projects (under framework and applications) that are all packaged up as one Eclipse project can be broken up into smaller projects that have a beautiful and meaningful dependency chain and still compile (at least). Executing them is probably another story. Now in terms of multi-tenant usage, I would love to be able to run different "instances" of OfBiz within the same server. So every tenant would get its own OfBiz configuration (including its own data source configuration). OfBiz features would actually be something additional, integrated with another application and OfBiz would not be the leading infastructure. I remember having seen some similar discussions on the mailing list some time ago - they all seemed to go nowhere. To me it seems that using OfBiz as the fundamental functional underpinning and toolbox for anything eCommerce is a very natural path to follow, but it should not have unbreakable ties to exactly one underlying infrastructure. I understand that OfBiz has spent some effort into creating its own module system (kind of) - that's perfectly fine and seeing OfBiz run out-of-the-box after 5min. is perfect. However, other places, other infrastructures. Anyway, if anybody has a pointer to something that is similar to what I described above or some success story, it would be great, if you could let me know. Of course, if there is good reasons why this cannot work w/o touching OfBiz in an unknown but huge number of places that would also be valuable information. If I end up getting it done myself and if there is interest, I will be happy to share my experience. Thanks, Henning |
We have made multi-tenant modifications to ofbiz to run on a single VM. We have not contributed this back to the community yet, but plan on it.
In our implementation, each tenant is running the same application suite, and they differ only in the database configurations via the entityengine.xml. We've essentially, made "templates" for the entity engine entries that allow each tenant to have their own delegator. This means, depending upon how you configure the entityengine.xml file, you can have each tenant have a private copy of the entire schema, or share components via the use of entity groups. There are some limitations in our design choices, that were appropriate for our use (for now): 1- We provide each tenant with a separate hostname, and then maintain a hostname to delegator name mapping in a "main" database instance. This is the deployment meta data. This is cumbersome and obviously has some extra dependancy on proper DNS configuration. This can be changed to a modified "login" page of some sort. 2- Haven't completed the job manager to have a singleton across tenants. Right now, the job manager is instantiated for each delegator, so, if you have 1000 tenants, you'd have a 1000 threats.. not scalable. Plan on fixing that. 3- Each tenant must share the same configuration files, means each tenant must be running same "application" or solution. 4- Currently each UserLogin entity is a private entity per tenant. Makes for improved isolation, but harder to manage all the login credentials to the system. We have an LDAP infrastructure and have debated using that for credential management. ----- Original Message ----- From: "Henning" <[hidden email]> To: [hidden email] Sent: Thursday, July 2, 2009 7:28:21 AM GMT -05:00 US/Canada Eastern Subject: Using OfBiz modular, partially, and in many instances Hi Experts, I would like to embed OfBiz with a module environment (not OSGi - but sufficiently similar to think OSGi if that helps) and for multiple tenants on one VM. The latter is the the stronger requirement for me. But as far as modularization is concerned, Ofbiz seems to be in good shape: The individual logical projects (under framework and applications) that are all packaged up as one Eclipse project can be broken up into smaller projects that have a beautiful and meaningful dependency chain and still compile (at least). Executing them is probably another story. Now in terms of multi-tenant usage, I would love to be able to run different "instances" of OfBiz within the same server. So every tenant would get its own OfBiz configuration (including its own data source configuration). OfBiz features would actually be something additional, integrated with another application and OfBiz would not be the leading infastructure. I remember having seen some similar discussions on the mailing list some time ago - they all seemed to go nowhere. To me it seems that using OfBiz as the fundamental functional underpinning and toolbox for anything eCommerce is a very natural path to follow, but it should not have unbreakable ties to exactly one underlying infrastructure. I understand that OfBiz has spent some effort into creating its own module system (kind of) - that's perfectly fine and seeing OfBiz run out-of-the-box after 5min. is perfect. However, other places, other infrastructures. Anyway, if anybody has a pointer to something that is similar to what I described above or some success story, it would be great, if you could let me know. Of course, if there is good reasons why this cannot work w/o touching OfBiz in an unknown but huge number of places that would also be valuable information. If I end up getting it done myself and if there is interest, I will be happy to share my experience. Thanks, Henning |
Marc,
that definitely sounds close to what I am trying to achieve. In case you do indeed go about making those modifications available, I would certainly volunteer to review and provide feedback. Thanks, Henning Am Donnerstag, den 02.07.2009, 09:31 -0400 schrieb Marc Morin: > We have made multi-tenant modifications to ofbiz to run on a single VM. We have not contributed this back to the community yet, but plan on it. > > In our implementation, each tenant is running the same application suite, and they differ only in the database configurations via the entityengine.xml. We've essentially, made "templates" for the entity engine entries that allow each tenant to have their own delegator. This means, depending upon how you configure the entityengine.xml file, you can have each tenant have a private copy of the entire schema, or share components via the use of entity groups. > > There are some limitations in our design choices, that were appropriate for our use (for now): > > 1- We provide each tenant with a separate hostname, and then maintain a hostname to delegator name mapping in a "main" database instance. This is the deployment meta data. This is cumbersome and obviously has some extra dependancy on proper DNS configuration. This can be changed to a modified "login" page of some sort. > 2- Haven't completed the job manager to have a singleton across tenants. Right now, the job manager is instantiated for each delegator, so, if you have 1000 tenants, you'd have a 1000 threats.. not scalable. Plan on fixing that. > 3- Each tenant must share the same configuration files, means each tenant must be running same "application" or solution. > 4- Currently each UserLogin entity is a private entity per tenant. Makes for improved isolation, but harder to manage all the login credentials to the system. We have an LDAP infrastructure and have debated using that for credential management. > > ----- Original Message ----- > From: "Henning" <[hidden email]> > To: [hidden email] > Sent: Thursday, July 2, 2009 7:28:21 AM GMT -05:00 US/Canada Eastern > Subject: Using OfBiz modular, partially, and in many instances > > Hi Experts, > > I would like to embed OfBiz with a module environment (not OSGi - but > sufficiently similar to think OSGi if that helps) and for multiple > tenants on one VM. The latter is the the stronger requirement for me. > But as far as modularization is concerned, Ofbiz seems to be in good > shape: The individual logical projects (under framework and > applications) that are all packaged up as one Eclipse project can be > broken up into smaller projects that have a beautiful and meaningful > dependency chain and still compile (at least). Executing them is > probably another story. > > Now in terms of multi-tenant usage, I would love to be able to run > different "instances" of OfBiz within the same server. So every tenant > would get its own OfBiz configuration (including its own data source > configuration). OfBiz features would actually be something additional, > integrated with another application and OfBiz would not be the leading > infastructure. > > I remember having seen some similar discussions on the mailing list > some time ago - they all seemed to go nowhere. To me it seems that using > OfBiz as the fundamental functional underpinning and toolbox for > anything eCommerce is a very natural path to follow, but it should not > have unbreakable ties to exactly one underlying infrastructure. I > understand that OfBiz has spent some effort into creating its own module > system (kind of) - that's perfectly fine and seeing OfBiz run > out-of-the-box after 5min. is perfect. However, other places, other > infrastructures. > > Anyway, if anybody has a pointer to something that is similar to what > I described above or some success story, it would be great, if you could > let me know. Of course, if there is good reasons why this cannot work > w/o touching OfBiz in an unknown but huge number of places that would > also be valuable information. > > If I end up getting it done myself and if there is interest, I will be > happy to share my experience. > > Thanks, > Henning |
In reply to this post by gnomie
Thanks for writing more about this Henning (and you too Marc in your reply). This goes back to what has been one of the toughest parts of OFBiz from day 1: the infrastructure to build it on. I'll admit I had only been using more "standard" J2EE stuff for only a couple of years before deciding to diverge from it. The lower level stuff is mostly fine (though it would be nice if JDBC implementations, aka drivers, where more consistent), but once you get above things like JDBC, JTA, Servlets, etc things get pretty messy (and ugly IMO) in the Java standards world. I guess fortunately they did split out these lower level standards and build the higher ones on top of them so that we can at least use them directly when the higher level ones don't fit the bill. The OFBiz component stuff was based on the concept of EAR files, but created because EAR files leave a lot to be desired. If there was a way to extend EAR files with some sort of plugin they might do the job, but as-is even app servers (inconsistently) feel the need to create extensions to it in order to get just the basic job done. The ofbiz-component.xsd file describes just what we want to configure in OFBiz for each component, though it is important to note that this changes over time as we add new things, and in general it's nice to be able to "plug-in" things since that's the only way to really make components independent and create generic tools that reduce dependencies between the components, at least on the configuration side of things. Of course, the OFBiz component stuff doesn't have a plugin mechanism either... we just take the easy way out of changing it when the need arises (usually when something very new is added to the framework). Back to multi-tenancy: if there was a good standard (even if not implemented yet) that we could use for it that would be great, but I'm not sure that such a thing exists, or at least not in a form that would be helpful. There certainly are things to facilitate running multiple instances on a single server, even machine and OS level virtualization, but those are inefficient and don't scale the way I think people are talking about (ie thousands of lightly used instances on a single server or pool of servers acting as one). So, my guess is that we'll have to do something custom for this. Anyway, something custom may make the most sense. With the entity engine we could build something (like Marc, Harmeet, and others at Emforium have done), and I like the idea of doing it so the user interaction is like what the big ones do, ie determine your instance by user, and that should be quite doable with a few variations (like putting UserLogin and certain other entities in a separate database (with their own entity-group)). Some newer features should help with this too, like the "configurable screen" (aka "portal") stuff, and the proposed security changes to externalize permissions from implementation artifacts. -David On Jul 2, 2009, at 5:28 AM, Henning wrote: > Hi Experts, > > I would like to embed OfBiz with a module environment (not OSGi - but > sufficiently similar to think OSGi if that helps) and for multiple > tenants on one VM. The latter is the the stronger requirement for me. > But as far as modularization is concerned, Ofbiz seems to be in good > shape: The individual logical projects (under framework and > applications) that are all packaged up as one Eclipse project can be > broken up into smaller projects that have a beautiful and meaningful > dependency chain and still compile (at least). Executing them is > probably another story. > > Now in terms of multi-tenant usage, I would love to be able to run > different "instances" of OfBiz within the same server. So every tenant > would get its own OfBiz configuration (including its own data source > configuration). OfBiz features would actually be something additional, > integrated with another application and OfBiz would not be the leading > infastructure. > > I remember having seen some similar discussions on the mailing list > some time ago - they all seemed to go nowhere. To me it seems that > using > OfBiz as the fundamental functional underpinning and toolbox for > anything eCommerce is a very natural path to follow, but it should not > have unbreakable ties to exactly one underlying infrastructure. I > understand that OfBiz has spent some effort into creating its own > module > system (kind of) - that's perfectly fine and seeing OfBiz run > out-of-the-box after 5min. is perfect. However, other places, other > infrastructures. > > Anyway, if anybody has a pointer to something that is similar to what > I described above or some success story, it would be great, if you > could > let me know. Of course, if there is good reasons why this cannot work > w/o touching OfBiz in an unknown but huge number of places that would > also be valuable information. > > If I end up getting it done myself and if there is interest, I will > be > happy to share my experience. > > Thanks, > Henning |
David,
thanks a lot for that background information - I am still somewhat new to OfBiz and for me this is great help for understanding. Personally I wouldn't bet on EARs or anything else very specific, as it is not clear wheter the whole industry - or the Java parts of it at least - will really buy into it. All Java EE servers have their own extensibility mechanism and unless you have one very monolithic application you will have to implement vendor-specific interfaces / declarations / packaging. OSGi looks like it's going to get quite far in terms of defining some lowest common denominator of a standardized module runtime system - but then who knows really? So, that said, there's no disagreement w.r.t. what you decided to do about OfBiz's infrastructure. In most instances (I think) the best approach is to make sure whatever you implement can (in principle - say w/o deep code changes) be integrated as a library, e.g. like you can embed Jetty in your otherwise completely regular Java app. So that the environment specific adaptation (e.g. classpath metadata and service registration in OSGi) can all be put into an adapter. For example, I think there should be nothing wrong with using OfBiz in a Desktop app. From what I can tell from experimenting around OfBiz is not far from there. Thanks, henning Am Sonntag, den 05.07.2009, 15:25 -0600 schrieb David E Jones: > Thanks for writing more about this Henning (and you too Marc in your > reply). This goes back to what has been one of the toughest parts of > OFBiz from day 1: the infrastructure to build it on. > > I'll admit I had only been using more "standard" J2EE stuff for only a > couple of years before deciding to diverge from it. The lower level > stuff is mostly fine (though it would be nice if JDBC implementations, > aka drivers, where more consistent), but once you get above things > like JDBC, JTA, Servlets, etc things get pretty messy (and ugly IMO) > in the Java standards world. I guess fortunately they did split out > these lower level standards and build the higher ones on top of them > so that we can at least use them directly when the higher level ones > don't fit the bill. > > The OFBiz component stuff was based on the concept of EAR files, but > created because EAR files leave a lot to be desired. If there was a > way to extend EAR files with some sort of plugin they might do the > job, but as-is even app servers (inconsistently) feel the need to > create extensions to it in order to get just the basic job done. The > ofbiz-component.xsd file describes just what we want to configure in > OFBiz for each component, though it is important to note that this > changes over time as we add new things, and in general it's nice to be > able to "plug-in" things since that's the only way to really make > components independent and create generic tools that reduce > dependencies between the components, at least on the configuration > side of things. Of course, the OFBiz component stuff doesn't have a > plugin mechanism either... we just take the easy way out of changing > it when the need arises (usually when something very new is added to > the framework). > > Back to multi-tenancy: if there was a good standard (even if not > implemented yet) that we could use for it that would be great, but I'm > not sure that such a thing exists, or at least not in a form that > would be helpful. There certainly are things to facilitate running > multiple instances on a single server, even machine and OS level > virtualization, but those are inefficient and don't scale the way I > think people are talking about (ie thousands of lightly used instances > on a single server or pool of servers acting as one). So, my guess is > that we'll have to do something custom for this. > > Anyway, something custom may make the most sense. With the entity > engine we could build something (like Marc, Harmeet, and others at > Emforium have done), and I like the idea of doing it so the user > interaction is like what the big ones do, ie determine your instance > by user, and that should be quite doable with a few variations (like > putting UserLogin and certain other entities in a separate database > (with their own entity-group)). > > Some newer features should help with this too, like the "configurable > screen" (aka "portal") stuff, and the proposed security changes to > externalize permissions from implementation artifacts. > > -David > > > On Jul 2, 2009, at 5:28 AM, Henning wrote: > > > Hi Experts, > > > > I would like to embed OfBiz with a module environment (not OSGi - but > > sufficiently similar to think OSGi if that helps) and for multiple > > tenants on one VM. The latter is the the stronger requirement for me. > > But as far as modularization is concerned, Ofbiz seems to be in good > > shape: The individual logical projects (under framework and > > applications) that are all packaged up as one Eclipse project can be > > broken up into smaller projects that have a beautiful and meaningful > > dependency chain and still compile (at least). Executing them is > > probably another story. > > > > Now in terms of multi-tenant usage, I would love to be able to run > > different "instances" of OfBiz within the same server. So every tenant > > would get its own OfBiz configuration (including its own data source > > configuration). OfBiz features would actually be something additional, > > integrated with another application and OfBiz would not be the leading > > infastructure. > > > > I remember having seen some similar discussions on the mailing list > > some time ago - they all seemed to go nowhere. To me it seems that > > using > > OfBiz as the fundamental functional underpinning and toolbox for > > anything eCommerce is a very natural path to follow, but it should not > > have unbreakable ties to exactly one underlying infrastructure. I > > understand that OfBiz has spent some effort into creating its own > > module > > system (kind of) - that's perfectly fine and seeing OfBiz run > > out-of-the-box after 5min. is perfect. However, other places, other > > infrastructures. > > > > Anyway, if anybody has a pointer to something that is similar to what > > I described above or some success story, it would be great, if you > > could > > let me know. Of course, if there is good reasons why this cannot work > > w/o touching OfBiz in an unknown but huge number of places that would > > also be valuable information. > > > > If I end up getting it done myself and if there is interest, I will > > be > > happy to share my experience. > > > > Thanks, > > Henning > |
I created OSGi bundles of OFBiz and running the framework as well as
applications inside OSGi kernel (Eclipse Equinox). I just took stock ofbiz and created a OSGi bundle with any code changes in the OFBiz. I think it creating separate components for OFBiz applications can make it pretty modular. As Henning said by my goal was to embed the OFBiz inside the Eclipse RCP to create a Desktop application and integration was pretty cool except that I could not modularize the stock ofbiz in various application components at this moment. However, this should not be much difficult making some minor changes in the framework. I am in the process of setting up a Sanbox on sourceforge and commit the code for people to play with. Thanks, Raj Henning wrote: > David, > > thanks a lot for that background information - I am still somewhat new > to OfBiz and for me this is great help for understanding. > > Personally I wouldn't bet on EARs or anything else very specific, as > it is not clear wheter the whole industry - or the Java parts of it at > least - will really buy into it. All Java EE servers have their own > extensibility mechanism and unless you have one very monolithic > application you will have to implement vendor-specific interfaces / > declarations / packaging. OSGi looks like it's going to get quite far in > terms of defining some lowest common denominator of a standardized > module runtime system - but then who knows really? So, that said, > there's no disagreement w.r.t. what you decided to do about OfBiz's > infrastructure. > > In most instances (I think) the best approach is to make sure whatever > you implement can (in principle - say w/o deep code changes) be > integrated as a library, e.g. like you can embed Jetty in your otherwise > completely regular Java app. So that the environment specific adaptation > (e.g. classpath metadata and service registration in OSGi) can all be > put into an adapter. For example, I think there should be nothing wrong > with using OfBiz in a Desktop app. > > From what I can tell from experimenting around OfBiz is not far from > there. > > Thanks, > henning > > > > Am Sonntag, den 05.07.2009, 15:25 -0600 schrieb David E Jones: > > >> Thanks for writing more about this Henning (and you too Marc in your >> reply). This goes back to what has been one of the toughest parts of >> OFBiz from day 1: the infrastructure to build it on. >> >> I'll admit I had only been using more "standard" J2EE stuff for only a >> couple of years before deciding to diverge from it. The lower level >> stuff is mostly fine (though it would be nice if JDBC implementations, >> aka drivers, where more consistent), but once you get above things >> like JDBC, JTA, Servlets, etc things get pretty messy (and ugly IMO) >> in the Java standards world. I guess fortunately they did split out >> these lower level standards and build the higher ones on top of them >> so that we can at least use them directly when the higher level ones >> don't fit the bill. >> >> The OFBiz component stuff was based on the concept of EAR files, but >> created because EAR files leave a lot to be desired. If there was a >> way to extend EAR files with some sort of plugin they might do the >> job, but as-is even app servers (inconsistently) feel the need to >> create extensions to it in order to get just the basic job done. The >> ofbiz-component.xsd file describes just what we want to configure in >> OFBiz for each component, though it is important to note that this >> changes over time as we add new things, and in general it's nice to be >> able to "plug-in" things since that's the only way to really make >> components independent and create generic tools that reduce >> dependencies between the components, at least on the configuration >> side of things. Of course, the OFBiz component stuff doesn't have a >> plugin mechanism either... we just take the easy way out of changing >> it when the need arises (usually when something very new is added to >> the framework). >> >> Back to multi-tenancy: if there was a good standard (even if not >> implemented yet) that we could use for it that would be great, but I'm >> not sure that such a thing exists, or at least not in a form that >> would be helpful. There certainly are things to facilitate running >> multiple instances on a single server, even machine and OS level >> virtualization, but those are inefficient and don't scale the way I >> think people are talking about (ie thousands of lightly used instances >> on a single server or pool of servers acting as one). So, my guess is >> that we'll have to do something custom for this. >> >> Anyway, something custom may make the most sense. With the entity >> engine we could build something (like Marc, Harmeet, and others at >> Emforium have done), and I like the idea of doing it so the user >> interaction is like what the big ones do, ie determine your instance >> by user, and that should be quite doable with a few variations (like >> putting UserLogin and certain other entities in a separate database >> (with their own entity-group)). >> >> Some newer features should help with this too, like the "configurable >> screen" (aka "portal") stuff, and the proposed security changes to >> externalize permissions from implementation artifacts. >> >> -David >> >> >> On Jul 2, 2009, at 5:28 AM, Henning wrote: >> >> >>> Hi Experts, >>> >>> I would like to embed OfBiz with a module environment (not OSGi - but >>> sufficiently similar to think OSGi if that helps) and for multiple >>> tenants on one VM. The latter is the the stronger requirement for me. >>> But as far as modularization is concerned, Ofbiz seems to be in good >>> shape: The individual logical projects (under framework and >>> applications) that are all packaged up as one Eclipse project can be >>> broken up into smaller projects that have a beautiful and meaningful >>> dependency chain and still compile (at least). Executing them is >>> probably another story. >>> >>> Now in terms of multi-tenant usage, I would love to be able to run >>> different "instances" of OfBiz within the same server. So every tenant >>> would get its own OfBiz configuration (including its own data source >>> configuration). OfBiz features would actually be something additional, >>> integrated with another application and OfBiz would not be the leading >>> infastructure. >>> >>> I remember having seen some similar discussions on the mailing list >>> some time ago - they all seemed to go nowhere. To me it seems that >>> using >>> OfBiz as the fundamental functional underpinning and toolbox for >>> anything eCommerce is a very natural path to follow, but it should not >>> have unbreakable ties to exactly one underlying infrastructure. I >>> understand that OfBiz has spent some effort into creating its own >>> module >>> system (kind of) - that's perfectly fine and seeing OfBiz run >>> out-of-the-box after 5min. is perfect. However, other places, other >>> infrastructures. >>> >>> Anyway, if anybody has a pointer to something that is similar to what >>> I described above or some success story, it would be great, if you >>> could >>> let me know. Of course, if there is good reasons why this cannot work >>> w/o touching OfBiz in an unknown but huge number of places that would >>> also be valuable information. >>> >>> If I end up getting it done myself and if there is interest, I will >>> be >>> happy to share my experience. >>> >>> Thanks, >>> Henning >>> > > |
Thanks Raj, that's really very interesting. Are you planning on
committing a modularized version or the bundle wrapper around the whole of OfBiz? Thanks, Henning Am Montag, den 06.07.2009, 17:00 +0530 schrieb Raj Saini: > I created OSGi bundles of OFBiz and running the framework as well as > applications inside OSGi kernel (Eclipse Equinox). I just took stock > ofbiz and created a OSGi bundle with any code changes in the OFBiz. I > think it creating separate components for OFBiz applications can make it > pretty modular. > > As Henning said by my goal was to embed the OFBiz inside the Eclipse RCP > to create a Desktop application and integration was pretty cool except > that I could not modularize the stock ofbiz in various application > components at this moment. However, this should not be much difficult > making some minor changes in the framework. > > I am in the process of setting up a Sanbox on sourceforge and commit the > code for people to play with. > > Thanks, > > Raj > > > Henning wrote: > > David, > > > > thanks a lot for that background information - I am still somewhat new > > to OfBiz and for me this is great help for understanding. > > > > Personally I wouldn't bet on EARs or anything else very specific, as > > it is not clear wheter the whole industry - or the Java parts of it at > > least - will really buy into it. All Java EE servers have their own > > extensibility mechanism and unless you have one very monolithic > > application you will have to implement vendor-specific interfaces / > > declarations / packaging. OSGi looks like it's going to get quite far in > > terms of defining some lowest common denominator of a standardized > > module runtime system - but then who knows really? So, that said, > > there's no disagreement w.r.t. what you decided to do about OfBiz's > > infrastructure. > > > > In most instances (I think) the best approach is to make sure whatever > > you implement can (in principle - say w/o deep code changes) be > > integrated as a library, e.g. like you can embed Jetty in your otherwise > > completely regular Java app. So that the environment specific adaptation > > (e.g. classpath metadata and service registration in OSGi) can all be > > put into an adapter. For example, I think there should be nothing wrong > > with using OfBiz in a Desktop app. > > > > From what I can tell from experimenting around OfBiz is not far from > > there. > > > > Thanks, > > henning > > > > > > > > Am Sonntag, den 05.07.2009, 15:25 -0600 schrieb David E Jones: > > > > > >> Thanks for writing more about this Henning (and you too Marc in your > >> reply). This goes back to what has been one of the toughest parts of > >> OFBiz from day 1: the infrastructure to build it on. > >> > >> I'll admit I had only been using more "standard" J2EE stuff for only a > >> couple of years before deciding to diverge from it. The lower level > >> stuff is mostly fine (though it would be nice if JDBC implementations, > >> aka drivers, where more consistent), but once you get above things > >> like JDBC, JTA, Servlets, etc things get pretty messy (and ugly IMO) > >> in the Java standards world. I guess fortunately they did split out > >> these lower level standards and build the higher ones on top of them > >> so that we can at least use them directly when the higher level ones > >> don't fit the bill. > >> > >> The OFBiz component stuff was based on the concept of EAR files, but > >> created because EAR files leave a lot to be desired. If there was a > >> way to extend EAR files with some sort of plugin they might do the > >> job, but as-is even app servers (inconsistently) feel the need to > >> create extensions to it in order to get just the basic job done. The > >> ofbiz-component.xsd file describes just what we want to configure in > >> OFBiz for each component, though it is important to note that this > >> changes over time as we add new things, and in general it's nice to be > >> able to "plug-in" things since that's the only way to really make > >> components independent and create generic tools that reduce > >> dependencies between the components, at least on the configuration > >> side of things. Of course, the OFBiz component stuff doesn't have a > >> plugin mechanism either... we just take the easy way out of changing > >> it when the need arises (usually when something very new is added to > >> the framework). > >> > >> Back to multi-tenancy: if there was a good standard (even if not > >> implemented yet) that we could use for it that would be great, but I'm > >> not sure that such a thing exists, or at least not in a form that > >> would be helpful. There certainly are things to facilitate running > >> multiple instances on a single server, even machine and OS level > >> virtualization, but those are inefficient and don't scale the way I > >> think people are talking about (ie thousands of lightly used instances > >> on a single server or pool of servers acting as one). So, my guess is > >> that we'll have to do something custom for this. > >> > >> Anyway, something custom may make the most sense. With the entity > >> engine we could build something (like Marc, Harmeet, and others at > >> Emforium have done), and I like the idea of doing it so the user > >> interaction is like what the big ones do, ie determine your instance > >> by user, and that should be quite doable with a few variations (like > >> putting UserLogin and certain other entities in a separate database > >> (with their own entity-group)). > >> > >> Some newer features should help with this too, like the "configurable > >> screen" (aka "portal") stuff, and the proposed security changes to > >> externalize permissions from implementation artifacts. > >> > >> -David > >> > >> > >> On Jul 2, 2009, at 5:28 AM, Henning wrote: > >> > >> > >>> Hi Experts, > >>> > >>> I would like to embed OfBiz with a module environment (not OSGi - but > >>> sufficiently similar to think OSGi if that helps) and for multiple > >>> tenants on one VM. The latter is the the stronger requirement for me. > >>> But as far as modularization is concerned, Ofbiz seems to be in good > >>> shape: The individual logical projects (under framework and > >>> applications) that are all packaged up as one Eclipse project can be > >>> broken up into smaller projects that have a beautiful and meaningful > >>> dependency chain and still compile (at least). Executing them is > >>> probably another story. > >>> > >>> Now in terms of multi-tenant usage, I would love to be able to run > >>> different "instances" of OfBiz within the same server. So every tenant > >>> would get its own OfBiz configuration (including its own data source > >>> configuration). OfBiz features would actually be something additional, > >>> integrated with another application and OfBiz would not be the leading > >>> infastructure. > >>> > >>> I remember having seen some similar discussions on the mailing list > >>> some time ago - they all seemed to go nowhere. To me it seems that > >>> using > >>> OfBiz as the fundamental functional underpinning and toolbox for > >>> anything eCommerce is a very natural path to follow, but it should not > >>> have unbreakable ties to exactly one underlying infrastructure. I > >>> understand that OfBiz has spent some effort into creating its own > >>> module > >>> system (kind of) - that's perfectly fine and seeing OfBiz run > >>> out-of-the-box after 5min. is perfect. However, other places, other > >>> infrastructures. > >>> > >>> Anyway, if anybody has a pointer to something that is similar to what > >>> I described above or some success story, it would be great, if you > >>> could > >>> let me know. Of course, if there is good reasons why this cannot work > >>> w/o touching OfBiz in an unknown but huge number of places that would > >>> also be valuable information. > >>> > >>> If I end up getting it done myself and if there is interest, I will > >>> be > >>> happy to share my experience. > >>> > >>> Thanks, > >>> Henning > >>> > > > > > |
Hi Henning,
Right now it is just one bundle wrapper and all the dependencies as OSGi bundles. Creating one or more bundle shouldn't be difficult though it will need some minor changes in the framework to resolve classpath issues. Thanks, Raj Henning wrote: > Thanks Raj, that's really very interesting. Are you planning on > committing a modularized version or the bundle wrapper around the whole > of OfBiz? > > Thanks, > Henning > > > Am Montag, den 06.07.2009, 17:00 +0530 schrieb Raj Saini: > > >> I created OSGi bundles of OFBiz and running the framework as well as >> applications inside OSGi kernel (Eclipse Equinox). I just took stock >> ofbiz and created a OSGi bundle with any code changes in the OFBiz. I >> think it creating separate components for OFBiz applications can make it >> pretty modular. >> >> As Henning said by my goal was to embed the OFBiz inside the Eclipse RCP >> to create a Desktop application and integration was pretty cool except >> that I could not modularize the stock ofbiz in various application >> components at this moment. However, this should not be much difficult >> making some minor changes in the framework. >> >> I am in the process of setting up a Sanbox on sourceforge and commit the >> code for people to play with. >> >> Thanks, >> >> Raj >> >> >> Henning wrote: >> >>> David, >>> >>> thanks a lot for that background information - I am still somewhat new >>> to OfBiz and for me this is great help for understanding. >>> >>> Personally I wouldn't bet on EARs or anything else very specific, as >>> it is not clear wheter the whole industry - or the Java parts of it at >>> least - will really buy into it. All Java EE servers have their own >>> extensibility mechanism and unless you have one very monolithic >>> application you will have to implement vendor-specific interfaces / >>> declarations / packaging. OSGi looks like it's going to get quite far in >>> terms of defining some lowest common denominator of a standardized >>> module runtime system - but then who knows really? So, that said, >>> there's no disagreement w.r.t. what you decided to do about OfBiz's >>> infrastructure. >>> >>> In most instances (I think) the best approach is to make sure whatever >>> you implement can (in principle - say w/o deep code changes) be >>> integrated as a library, e.g. like you can embed Jetty in your otherwise >>> completely regular Java app. So that the environment specific adaptation >>> (e.g. classpath metadata and service registration in OSGi) can all be >>> put into an adapter. For example, I think there should be nothing wrong >>> with using OfBiz in a Desktop app. >>> >>> From what I can tell from experimenting around OfBiz is not far from >>> there. >>> >>> Thanks, >>> henning >>> >>> >>> >>> Am Sonntag, den 05.07.2009, 15:25 -0600 schrieb David E Jones: >>> >>> >>> >>>> Thanks for writing more about this Henning (and you too Marc in your >>>> reply). This goes back to what has been one of the toughest parts of >>>> OFBiz from day 1: the infrastructure to build it on. >>>> >>>> I'll admit I had only been using more "standard" J2EE stuff for only a >>>> couple of years before deciding to diverge from it. The lower level >>>> stuff is mostly fine (though it would be nice if JDBC implementations, >>>> aka drivers, where more consistent), but once you get above things >>>> like JDBC, JTA, Servlets, etc things get pretty messy (and ugly IMO) >>>> in the Java standards world. I guess fortunately they did split out >>>> these lower level standards and build the higher ones on top of them >>>> so that we can at least use them directly when the higher level ones >>>> don't fit the bill. >>>> >>>> The OFBiz component stuff was based on the concept of EAR files, but >>>> created because EAR files leave a lot to be desired. If there was a >>>> way to extend EAR files with some sort of plugin they might do the >>>> job, but as-is even app servers (inconsistently) feel the need to >>>> create extensions to it in order to get just the basic job done. The >>>> ofbiz-component.xsd file describes just what we want to configure in >>>> OFBiz for each component, though it is important to note that this >>>> changes over time as we add new things, and in general it's nice to be >>>> able to "plug-in" things since that's the only way to really make >>>> components independent and create generic tools that reduce >>>> dependencies between the components, at least on the configuration >>>> side of things. Of course, the OFBiz component stuff doesn't have a >>>> plugin mechanism either... we just take the easy way out of changing >>>> it when the need arises (usually when something very new is added to >>>> the framework). >>>> >>>> Back to multi-tenancy: if there was a good standard (even if not >>>> implemented yet) that we could use for it that would be great, but I'm >>>> not sure that such a thing exists, or at least not in a form that >>>> would be helpful. There certainly are things to facilitate running >>>> multiple instances on a single server, even machine and OS level >>>> virtualization, but those are inefficient and don't scale the way I >>>> think people are talking about (ie thousands of lightly used instances >>>> on a single server or pool of servers acting as one). So, my guess is >>>> that we'll have to do something custom for this. >>>> >>>> Anyway, something custom may make the most sense. With the entity >>>> engine we could build something (like Marc, Harmeet, and others at >>>> Emforium have done), and I like the idea of doing it so the user >>>> interaction is like what the big ones do, ie determine your instance >>>> by user, and that should be quite doable with a few variations (like >>>> putting UserLogin and certain other entities in a separate database >>>> (with their own entity-group)). >>>> >>>> Some newer features should help with this too, like the "configurable >>>> screen" (aka "portal") stuff, and the proposed security changes to >>>> externalize permissions from implementation artifacts. >>>> >>>> -David >>>> >>>> >>>> On Jul 2, 2009, at 5:28 AM, Henning wrote: >>>> >>>> >>>> >>>>> Hi Experts, >>>>> >>>>> I would like to embed OfBiz with a module environment (not OSGi - but >>>>> sufficiently similar to think OSGi if that helps) and for multiple >>>>> tenants on one VM. The latter is the the stronger requirement for me. >>>>> But as far as modularization is concerned, Ofbiz seems to be in good >>>>> shape: The individual logical projects (under framework and >>>>> applications) that are all packaged up as one Eclipse project can be >>>>> broken up into smaller projects that have a beautiful and meaningful >>>>> dependency chain and still compile (at least). Executing them is >>>>> probably another story. >>>>> >>>>> Now in terms of multi-tenant usage, I would love to be able to run >>>>> different "instances" of OfBiz within the same server. So every tenant >>>>> would get its own OfBiz configuration (including its own data source >>>>> configuration). OfBiz features would actually be something additional, >>>>> integrated with another application and OfBiz would not be the leading >>>>> infastructure. >>>>> >>>>> I remember having seen some similar discussions on the mailing list >>>>> some time ago - they all seemed to go nowhere. To me it seems that >>>>> using >>>>> OfBiz as the fundamental functional underpinning and toolbox for >>>>> anything eCommerce is a very natural path to follow, but it should not >>>>> have unbreakable ties to exactly one underlying infrastructure. I >>>>> understand that OfBiz has spent some effort into creating its own >>>>> module >>>>> system (kind of) - that's perfectly fine and seeing OfBiz run >>>>> out-of-the-box after 5min. is perfect. However, other places, other >>>>> infrastructures. >>>>> >>>>> Anyway, if anybody has a pointer to something that is similar to what >>>>> I described above or some success story, it would be great, if you >>>>> could >>>>> let me know. Of course, if there is good reasons why this cannot work >>>>> w/o touching OfBiz in an unknown but huge number of places that would >>>>> also be valuable information. >>>>> >>>>> If I end up getting it done myself and if there is interest, I will >>>>> be >>>>> happy to share my experience. >>>>> >>>>> Thanks, >>>>> Henning >>>>> >>>>> >>> >>> > > |
In reply to this post by David E. Jones-2
After spending some more time on stepping through the OfBiz code and
trying to get the entity engine running outside of the OfBiz framework I found that while you can run the entity engine like that, but it still requires a single entityengine.xml from the classpath and will maintain that in some static cache. I think I saw the same pattern applied for other configurations. In my understanding that means that one instance of the entity engine code can be used for exactly one configuration per VM and in particular, for the multi-tenant use-case, all data sources for all tenants have to be pre-defined in that one configuration file. If I understood Marc correctly, that's also what he does. So the entity engine can be configured exactly once and it will not forget until the VM stops. However, it seems that all you need to process a request for a tenant is its delegator and subsequently a service dispatcher. Both, it seems, could be created based on some tenant specific configuration and still work correctly, if it wasn't for the static cache. Also, it seems that the ResourceLoader supports separation per class loader, so that you could have different entityengine.xml files in different deployable (assuming these use different classloaders at runtime) and the one to be the first one to ask for a delegator would actually provide the entity engine config. Unfortunately any other code would then share that configuration. Plus... GenericDelegator, EntityConfigUtil etc. would not know about different classloaders still. Currently I have the impression that trying to change all those places could be an endless endeavour and I better start looking into a solution that - at runtime - has duplicate deployments of OfBiz. Any thoughts? Thanks, Henning |
In reply to this post by gnomie
We do indeed have a single copy of the entityengine.xml file, but we have added "templates" to the file, such that we have an entity in our main "provisioning" instance, that lists all the tenants and the coordinates of the database for their delegator.
The delegator for the tenants are then instantiated using the template and the information from this provisioning database. This means, that in a VM, all instances share the same "structure" of delegator and data sources. Below is a snippet of our entityengine.xml file that illustrates the template delegator and associated datasource. The FlexibleStringExpander is used to expand the template items. Our delegators have names that are numbers, (ie. party_id of our customer in our deployment instance). Each tenant has a row in the DatabaseDetail entity which has columns databaseName, internalHost, externalHost. In our case, the username and password are generated (not stored in the database). The internalHost is the internal hostname for the database server, while the externalHost is the external hostname for the database server. Since we deploy on Amazon EC2, each host has both a local and public IP address. If your paying attention to the information below, you can see that the sequence numbers for each tenant are prefixed with the party_id and a "-". Making all surrogate keys non-colliding (an important feature for sharing database between tenants). <delegator name="default" entity-model-reader="main" entity-group-reader="main" entity-eca-reader="main" distributed-cache-clear-enabled="false"> <group-map group-name="org.ofbiz" datasource-name="localpostgres"/> <group-map group-name="org.ofbiz.olap" datasource-name="localpostgres"/> <group-map group-name="com.emforium.deploy" datasource-name="emforium"/> </delegator> <delegator name="${instance}" crypto-minimum-keys="0" entity-model-reader="main" entity-group-reader="main" entity-eca-reader="main" distributed-cache-clear-enabled="false" sequenced-id-prefix="${instance}-"> <group-map group-name="org.ofbiz" datasource-name="${instance}"/> <group-map group-name="org.ofbiz.olap" datasource-name="${instance}"/> <group-map group-name="com.emforium.deploy" datasource-name="${instance}"/> </delegator> <datasource name="${instance}" helper-class="org.ofbiz.entity.datasource.GenericHelperDAO" schema-name="public" field-type-name="postgres" check-on-start="true" add-missing-on-start="true" generate-sql="false" use-fk-initially-deferred="false" alias-view-columns="false" join-style="ansi" use-binary-type-for-blob="true"> <after-create-table>GRANT SELECT,INSERT,UPDATE,DELETE ON TABLE ${table} TO runtime</after-create-table> <after-create-table>GRANT SELECT ON TABLE ${table} TO readonly</after-create-table> <after-create-table>REVOKE ALL ON TABLE ${table} FROM public</after-create-table> <after-create-table>ALTER TABLE ${table} OWNER TO admin</after-create-table> <after-create-view>GRANT SELECT ON TABLE ${table} TO runtime</after-create-view> <after-create-view>GRANT SELECT ON TABLE ${table} TO readonly</after-create-view> <after-create-view>REVOKE ALL ON TABLE ${table} FROM public</after-create-view> <after-create-view>ALTER TABLE ${table} OWNER TO admin</after-create-view> <inline-jdbc jdbc-driver="org.postgresql.Driver" jdbc-uri="jdbc:postgresql://${internalHost}/${databaseName}" jdbc-username="${username}" jdbc-password="${password}" isolation-level="ReadCommitted" pool-minsize="2" pool-maxsize="250"/> </datasource> ----- Original Message ----- From: "Henning" <[hidden email]> To: [hidden email] Sent: Wednesday, July 8, 2009 6:25:15 AM GMT -05:00 US/Canada Eastern Subject: Re: Using OfBiz modular, partially, and in many instances After spending some more time on stepping through the OfBiz code and trying to get the entity engine running outside of the OfBiz framework I found that while you can run the entity engine like that, but it still requires a single entityengine.xml from the classpath and will maintain that in some static cache. I think I saw the same pattern applied for other configurations. In my understanding that means that one instance of the entity engine code can be used for exactly one configuration per VM and in particular, for the multi-tenant use-case, all data sources for all tenants have to be pre-defined in that one configuration file. If I understood Marc correctly, that's also what he does. So the entity engine can be configured exactly once and it will not forget until the VM stops. However, it seems that all you need to process a request for a tenant is its delegator and subsequently a service dispatcher. Both, it seems, could be created based on some tenant specific configuration and still work correctly, if it wasn't for the static cache. Also, it seems that the ResourceLoader supports separation per class loader, so that you could have different entityengine.xml files in different deployable (assuming these use different classloaders at runtime) and the one to be the first one to ask for a delegator would actually provide the entity engine config. Unfortunately any other code would then share that configuration. Plus... GenericDelegator, EntityConfigUtil etc. would not know about different classloaders still. Currently I have the impression that trying to change all those places could be an endless endeavour and I better start looking into a solution that - at runtime - has duplicate deployments of OfBiz. Any thoughts? Thanks, Henning |
Marc,
thanks for that detailed update. I think you have come very far in terms of solving the real problems! After I sent that (slightly pessimistic) mail yesterday I started digging into the code once more and now think there are two reasonable possiblities to get tenant config separated - just for the goal of being able to move tenants without having to touch a lot of shared config files. a) Fabricate the entityengine.xml from tenant config that resides in tenant apps. That is a somewhat local change and just a simple extension to the template mechanism you have. b) Change those places where OfBiz keeps config in class variables so that its wrapped by a WeakHashMap<Classloader,T> so that we can separate config by the thread's context classloader. Somewhat a hack, but it will show what you need to maintain in a "tenant context" if you would want to make OfBiz really multi-tenant capable. However, what you describe below is extremely useful regardless of a) or b) (in particular making sure that keys don't collide). And I haven't even looked into operating the web apps in a multi-tenant scenario yet. I guess I need to experiment a little longer to get settled on a plan. Thanks, Henning Am Mittwoch, den 08.07.2009, 17:11 -0400 schrieb Marc Morin: > We do indeed have a single copy of the entityengine.xml file, but we have added "templates" to the file, such that we have an entity in our main "provisioning" instance, that lists all the tenants and the coordinates of the database for their delegator. > > The delegator for the tenants are then instantiated using the template and the information from this provisioning database. > > This means, that in a VM, all instances share the same "structure" of delegator and data sources. Below is a snippet of our entityengine.xml file that illustrates the template delegator and associated datasource. > > The FlexibleStringExpander is used to expand the template items. Our delegators have names that are numbers, (ie. party_id of our customer in our deployment instance). Each tenant has a row in the DatabaseDetail entity which has columns databaseName, internalHost, externalHost. In our case, the username and password are generated (not stored in the database). > > The internalHost is the internal hostname for the database server, while the externalHost is the external hostname for the database server. Since we deploy on Amazon EC2, each host has both a local and public IP address. > > If your paying attention to the information below, you can see that the sequence numbers for each tenant are prefixed with the party_id and a "-". Making all surrogate keys non-colliding (an important feature for sharing database between tenants). > > > <delegator name="default" entity-model-reader="main" entity-group-reader="main" entity-eca-reader="main" distributed-cache-clear-enabled="false"> > <group-map group-name="org.ofbiz" datasource-name="localpostgres"/> > <group-map group-name="org.ofbiz.olap" datasource-name="localpostgres"/> > <group-map group-name="com.emforium.deploy" datasource-name="emforium"/> > </delegator> > <delegator name="${instance}" crypto-minimum-keys="0" entity-model-reader="main" entity-group-reader="main" entity-eca-reader="main" distributed-cache-clear-enabled="false" sequenced-id-prefix="${instance}-"> > <group-map group-name="org.ofbiz" datasource-name="${instance}"/> > <group-map group-name="org.ofbiz.olap" datasource-name="${instance}"/> > <group-map group-name="com.emforium.deploy" datasource-name="${instance}"/> > </delegator> > > <datasource name="${instance}" > helper-class="org.ofbiz.entity.datasource.GenericHelperDAO" > schema-name="public" > field-type-name="postgres" > check-on-start="true" > add-missing-on-start="true" > generate-sql="false" > use-fk-initially-deferred="false" > alias-view-columns="false" > join-style="ansi" > use-binary-type-for-blob="true"> > > <after-create-table>GRANT SELECT,INSERT,UPDATE,DELETE ON TABLE ${table} TO runtime</after-create-table> > <after-create-table>GRANT SELECT ON TABLE ${table} TO readonly</after-create-table> > <after-create-table>REVOKE ALL ON TABLE ${table} FROM public</after-create-table> > <after-create-table>ALTER TABLE ${table} OWNER TO admin</after-create-table> > <after-create-view>GRANT SELECT ON TABLE ${table} TO runtime</after-create-view> > <after-create-view>GRANT SELECT ON TABLE ${table} TO readonly</after-create-view> > <after-create-view>REVOKE ALL ON TABLE ${table} FROM public</after-create-view> > <after-create-view>ALTER TABLE ${table} OWNER TO admin</after-create-view> > > <inline-jdbc > jdbc-driver="org.postgresql.Driver" > jdbc-uri="jdbc:postgresql://${internalHost}/${databaseName}" > jdbc-username="${username}" > jdbc-password="${password}" > isolation-level="ReadCommitted" > pool-minsize="2" > pool-maxsize="250"/> > </datasource> > > > ----- Original Message ----- > From: "Henning" <[hidden email]> > To: [hidden email] > Sent: Wednesday, July 8, 2009 6:25:15 AM GMT -05:00 US/Canada Eastern > Subject: Re: Using OfBiz modular, partially, and in many instances > > After spending some more time on stepping through the OfBiz code and > trying to get the entity engine running outside of the OfBiz framework I > found that while you can run the entity engine like that, but it still > requires a single entityengine.xml from the classpath and will maintain > that in some static cache. I think I saw the same pattern applied for > other configurations. > > In my understanding that means that one instance of the entity engine > code can be used for exactly one configuration per VM and in particular, > for the multi-tenant use-case, all data sources for all tenants have to > be pre-defined in that one configuration file. If I understood Marc > correctly, that's also what he does. > > So the entity engine can be configured exactly once and it will not > forget until the VM stops. > > However, it seems that all you need to process a request for a tenant is > its delegator and subsequently a service dispatcher. Both, it seems, > could be created based on some tenant specific configuration and still > work correctly, if it wasn't for the static cache. > > Also, it seems that the ResourceLoader supports separation per class > loader, so that you could have different entityengine.xml files in > different deployable (assuming these use different classloaders at > runtime) and the one to be the first one to ask for a delegator would > actually provide the entity engine config. Unfortunately any other code > would then share that configuration. Plus... GenericDelegator, > EntityConfigUtil etc. would not know about different classloaders still. > > Currently I have the impression that trying to change all those places > could be an endless endeavour and I better start looking into a solution > that - at runtime - has duplicate deployments of OfBiz. > > Any thoughts? > > Thanks, > Henning > > |
Free forum by Nabble | Edit this page |