[GitHub] [ofbiz-framework] danwatford opened a new pull request #246: Improved: Configure database driver and data source from gradle property (OFBIZ-12081)
Locked
12
12
 
|
danwatford opened a new pull request #246: URL: https://github.com/apache/ofbiz-framework/pull/246 These changes proposed to support the possibility of ofbiz building docker images that target various DBMS. By having the DBMS configured as a property we can ensure an 'approved' version of the relevant JDBC driver is downloaded and added to the runtime classpath. If the user wishes to control which database library is used they can leave the property blank. The same gradle property also drives behaviour of a new build task, updateEntityEngine, which will modify the datasource values of the delegators specified in entityengine.xml based on the chosen DBMS. The potential result of these changes are a reduction in configuration steps for simple deployment cases, particularly those where ofbiz and its database run in docker containers as part of a docker-compose application. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
|
|
In reply to this post by GitBox
mbrohl commented on pull request #246: URL: https://github.com/apache/ofbiz-framework/pull/246#issuecomment-736335545 Hi Dan, why not provide all configuration properties instead of a database reference? The reference/prefix as well as the driver / version could be properties. This would make the Gradle code less cluttered by switch statements and additional code for every database flavour. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
|
|
In reply to this post by GitBox
danwatford commented on pull request #246: URL: https://github.com/apache/ofbiz-framework/pull/246#issuecomment-736348137 Hi @mbrohl I agree that we could provide a lot of database configuration via properties. Properties can be set in gradle.properties, via environment variables and on the gradlew commandline so there are lots of possibilities for production of pre-configured builds. For this PR I had intended to set the database username and password through properties and amend entityengine.xml accordingly, but ran into some issues where groovy had parsed those xml attributes as read-only, preventing me from updating them. My limited knowledge of groovy and XmlParser was probably the cause, but after losing several hours to something which should be simple I thought I'd better submit what I had so far and see if anyone else might be able to help. There are lots of things we could do to configure database access through build-time properties and amend entityengine.xml, but I didn't want to spend longer on it without some community buy-in to the idea. Regarding driver versioning, I liked the idea of a top-level property where the user choses their DBMS which then drives an opinionated selection of the driver version. Of course we would still honour the user's version preference if they give one, but if the user later requested support for database connectivity issues we would suggest they use the version recommended by the build system. If the approach is acceptable we can get these changes into trunk where I and others can iterate to improve and clean up the build script to incorporate properties deemed useful. I'd like to avoid trying to do too much as once otherwise I fear we'll end up with a long-lived branch with a single contributor trying to get the changes made when perhaps we would have benefited from more people using and incrementally improving the implementation. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
|
|
In reply to this post by GitBox
ieugen commented on pull request #246: URL: https://github.com/apache/ofbiz-framework/pull/246#issuecomment-736520552 I'm also against putting too much logic in gradle. In production environment, changing the configuration is something that is mandatory and that should be a simple file change or environment variable change. The jdbc drivers could also be pre-bundled for PostgreSQL and MySQL (MariaDB) - or at least instructions to easily download them. IMO, in this case having separate configuration files for each database is better. (postgres.smaple-cofig.xml that can be renamed when you install for example). ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
|
|
In reply to this post by GitBox
ieugen edited a comment on pull request #246: URL: https://github.com/apache/ofbiz-framework/pull/246#issuecomment-736520552 I'm also against putting too much logic in gradle. In production environment, changing the configuration is something that is mandatory and that should be a simple file change or environment variable change. The jdbc drivers could also be pre-bundled for PostgreSQL and MySQL (MariaDB) - or at least instructions to easily download them. IMO, in this case having separate configuration files for each database is better. (postgres.smaple-cofig.xml that can be renamed when you install or you can mount them inside docker with a different name.) ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
|
|
In reply to this post by GitBox
danwatford commented on pull request #246: URL: https://github.com/apache/ofbiz-framework/pull/246#issuecomment-736526172 Gradle was a useful location to put logic related to builds, but if build.gradle was getting a bit ugly we could spin code out to separate files. However I like the idea of DBMS specific configuration files. These files could be templates which are populated with username, password, etc, as build time based on property values, with suitable default values to fall back on. I'll try out the approach of copying and populating a template file at build time. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
|
|
In reply to this post by GitBox
ieugen commented on pull request #246: URL: https://github.com/apache/ofbiz-framework/pull/246#issuecomment-736535273 Glad it helps, but don't put too much in gradle. Gradle is a good tool and I love it but production people want to unpack a (single ?!) binary add a configuration file or command line options and run it. Complex tools just to generate a configuration VS plain old search and replace in text editor/ sed . In production you usually make your config once and tweak it over time while restarting the app. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
|
|
In reply to this post by GitBox
ieugen edited a comment on pull request #246: URL: https://github.com/apache/ofbiz-framework/pull/246#issuecomment-736535273 Glad it helps, but don't put too much in gradle. Gradle is a good tool and I love it but production people want to unpack a (single ?!) binary add a configuration file or command line options and run it. Complex tools just to generate a configuration VS plain old search and replace in text editor/ sed . In production you usually make your config once and tweak it over time while restarting the app. Also people can automate and manage from examples and instructions based on their needs and knowledge. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
|
|
In reply to this post by GitBox
ieugen edited a comment on pull request #246: URL: https://github.com/apache/ofbiz-framework/pull/246#issuecomment-736535273 Glad it helps, but don't put too much in gradle. Gradle is a good tool and I love it but production people want to unpack a (single ?!) binary add a configuration file or command line options and run it. Complex tools just to generate a configuration VS plain old search and replace in text editor/ sed . In production you usually make your config once and tweak it over time while restarting the app. Also people can automate and manage from examples and instructions based on their needs and knowledge. Bellow is an example from a Prometheus deployment with docker only. FYI it's very easy to translate such a deployment to kubernetes. Please note the use of volumes and mounted host directory, environment variables and command line arguments. ``` docker volume create prometheus-storage cd /opt/monitoring docker run --detach \ --name prometheus \ --restart unless-stopped \ --hostname prometheus \ --env VIRTUAL_PORT=9090 \ --volume prometheus-storage:/prometheus \ --volume /opt/monitoring/prometheus:/etc/prometheus:ro \ prom/prometheus:v2.22.1 \ --config.file=/etc/prometheus/prometheus.yml \ --storage.tsdb.path=/prometheus \ --storage.tsdb.retention.time=30d \ --web.console.libraries=/usr/share/prometheus/console_libraries \ --web.console.templates=/usr/share/prometheus/consoles ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
|
|
In reply to this post by GitBox
danwatford commented on pull request #246: URL: https://github.com/apache/ofbiz-framework/pull/246#issuecomment-736700620 Hi @ieugen , I've committed some changes to select and populate a template entityengine.xml file based property values. This nicely side-steps those problems I was having with Groovy's XmlParser and hopefully reduces the complexity of login in build.gradle down to ensuring suitable default values are in place for properties. I had envisioned these changes would be used to create docker images that would be used with DBMS example docker-compose specifications. Similar to your Prometheus example I would expect key directories and configuration files to be exposed on the host's filesystem allowing for local modification. I thought we might end up with docker-compose files similar to: ``` services: db: image: postgres:13 restart: always environment: POSTGRES_PASSWORD: lDvEFVkVyVpHuCOK9To3 OFBIZ_DB_USER: ofbizuser OFBIZ_DB_PASSWORD: ofbizpassword volumes: - ./dbdumps:/opt/dbdumps - ./docker-entrypoint-initdb.d:/docker-entrypoint-initdb.d ofbiz: build: ofbiz depends_on: - db ports: - 8443:8443 - 15005:5005 environment: ORG_GRADLE_PROJECT_ofbizDbms: postgres ORG_GRADLE_PROJECT_ofbizDbUsername: ofbizuser ORG_GRADLE_PROJECT_ofbizDbPassword: ofbizpassword ``` There are still some things to figure out. Perhaps we should avoid overwriting any existing entityengine.xml file if it exist, instead requiring an explicit call to the copyEntityEngineXml task to create a file which is then exposed by docker for editing. I also need to figure out the cause the CI build failure! @mbrohl , @ieugen Does it feel like this is going in the right direction? My motivation for this is to try and make docker images for both developer testing, demonstrations and something that could potentially be used in production behind a Traefik reverse-proxy. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
|
|
In reply to this post by GitBox
danwatford commented on pull request #246: URL: https://github.com/apache/ofbiz-framework/pull/246#issuecomment-758550984 Hi @mbrohl and @ieugen , Since I last looked at this piece of work I've become aware of the project's use of the gradle Application plugin to create distributions. This suggests that gradle might not always be used for deploying an ofbiz instance and therefore using gradle.properties to configure database access properties (username, password, etc) may not be all that useful. I do wonder though if there is still value in identifying approved versions of database drivers and selecting them for inclusion in a distribution (by adding to the dependencies list) based on a build-time property via the command line or gradle.properties. This would reduce the deployment process by one step since the administrator will not have to figure out which driver is declared as compatible with ofbiz for their particular DBMS. I'll drop the changes related to properties for DB access (username, password, etc) and query about the value of resolving a driver based on a build property to the mailing list. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
|
|
In reply to this post by GitBox
ieugen commented on pull request #246: URL: https://github.com/apache/ofbiz-framework/pull/246#issuecomment-758586824 Hi @danwatford , I think trying to meddle with those configs at build time is brittle and would complicate the build. I don't know about deploy via gradle, but to help with deploying via application plugin (ofbiz.tar) it would help to have the ability to add things to the classpath. If you could add this to `build.gradle` then people can add files to `config/` and `lib-extra/` directories and they will be available for ofbiz when it starts. I've tested this and it works for my ofbiz docker build (I'm planning an article these next 2 days and will share it). I'm also adding the database drivers post build to `lib-extra` . That way I can keep the ofbiz source unchanged and still get what I need. (I don't have windows to test the classpath ). ``` tasks.startScripts { doLast { // Alter the start script for Unix systems. unixScript.text = unixScript.text.replace('CLASSPATH=$APP_HOME/lib', 'CLASSPATH=$APP_HOME/config/:$APP_HOME/lib-extra/*:$APP_HOME/lib') // Alter the start script for Windows systems. // windowsScript.text = // windowsScript.text.replace('CLASSPATH=$APP_HOME/lib', // 'CLASSPATH=$APP_HOME\\conf\\:$APP_HOME/lib-extra/*:$APP_HOME/lib') } } ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
|
|
In reply to this post by GitBox
ieugen edited a comment on pull request #246: URL: https://github.com/apache/ofbiz-framework/pull/246#issuecomment-758586824 Hi @danwatford , I think trying to meddle with those configs at build time is brittle and would complicate the build. I don't know about deploy via gradle, but to help with deploying via application plugin (ofbiz.tar) it would help to have the ability to add things to the classpath. If you could add this to `build.gradle` then people can add files to `config/` and `lib-extra/` directories and they will be available for ofbiz when it starts. I've tested this and it works for my ofbiz docker build (I'm planning an article these next 2 days and will share it. Spoiler: it works on ARM - raspberry pi 4 ;) ). I'm also adding the database drivers post build to `lib-extra` . That way I can keep the ofbiz source unchanged and still get what I need. (I don't have windows to test the classpath ). ``` tasks.startScripts { doLast { // Alter the start script for Unix systems. unixScript.text = unixScript.text.replace('CLASSPATH=$APP_HOME/lib', 'CLASSPATH=$APP_HOME/config/:$APP_HOME/lib-extra/*:$APP_HOME/lib') // Alter the start script for Windows systems. // windowsScript.text = // windowsScript.text.replace('CLASSPATH=$APP_HOME/lib', // 'CLASSPATH=$APP_HOME\\conf\\:$APP_HOME/lib-extra/*:$APP_HOME/lib') } } ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
«
Return to OFBiz - Notifications
|
1 view|%1 views
| Free forum by Nabble | Edit this page |