Help Connecting to 2nd External Database

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Help Connecting to 2nd External Database

Jed Glazner
Hello,

I'm trying to connect to a 2nd external database (not opentaps) where I only have read only rights.  It's an adserver that powers the ads on our site.  
We only have read permissions and can't alter the tables at all.  our normal opentaps databse uses postgres, but the adserver uses MySQL.

What i've tried is to create a new datasource in the framework/entity/config/entityengine.xml.
I then wrote a service which uses the ConnectionFactory class.  The problem is that I get a Java Heap Space error the first time the service runs.
After that the service will run succesfully without any errors, until the next time ofbiz get started, then I get the heap space error once, and then it works fine again.  
I don't want to do just strait JDBC becauseI want to take advantage of the connection pooling within opentaps.

Any help would be appreciated.  The service and datasource are included.

-- datasource from entityengine.xml --

    <datasource name="mysql-advertpro"
            helper-class="org.ofbiz.entity.datasource.GenericHelperDAO"
            field-type-name="mysql"
            check-on-start="true"
            add-missing-on-start="false"
            check-pks-on-start="false"
            use-foreign-keys="true"
            join-style="ansi-no-parenthesis"
            alias-view-columns="false"
            drop-fk-use-foreign-key-keyword="true"
            table-type="MyISAM"
            character-set="latin1"
            collate="latin1_general_cs">
        <inline-jdbc
                jdbc-driver="com.mysql.jdbc.Driver"
                jdbc-uri="jdbc:mysql://<ipaddress>/advertpro?autoReconnect=true"
                jdbc-username="<username>"
                jdbc-password="<password>"
                isolation-level="ReadCommitted"
                pool-minsize="2"
                pool-maxsize="20"/>
    </datasource>

-- Custom Service --

public static Map getBannerCampaignDetails(DispatchContext ctx, Map context) {
                Connection conn = null;
                PreparedStatement statement = null;
                ResultSet rs = null;
                FastList bannerList = FastList.newInstance();
               
                String sql = "MySQL String Goes Here";
                               
                try{
                        conn = ConnectionFactory.getConnection("mysql-advertpro");
                        if (conn == null) {
                                throw new Exception("No advertpro connection configured");
                        }
                        statement = conn.prepareStatement(sql);
                        statement.setString(1, (String) context.get("partyId"));
                        rs = statement.executeQuery();
                        while(rs.next()){
                                FastMap row = FastMap.newInstance();
                                row.put("views", rs.getString("views"));
                                row.put("name", rs.getString("name"));
                                row.put("campaignId", rs.getString("campaignId"));
                                row.put("maxviews", rs.getString("maxviews"));
                                row.put("bannerType", rs.getString("bannerType"));
                                row.put("width", rs.getString("width"));
                                row.put("height", rs.getString("height"));
                                row.put("viewsRemaining", rs.getString("viewsRemaining"));
                                row.put("startdate", rs.getString("startdate"));
                                row.put("stopdate", rs.getString("stopdate"));
                                bannerList.add(row);
                        }
                        statement.close();
                        rs.close();
                } catch (Exception e){
                        ServiceUtil.returnError("Error getting connection to database" + e);
                }
               
                finally {
                        if (statement != null)
                                try {
                                        statement.close();
                                } catch (Exception e) {}
                        if (rs != null)
                                try {
                                        rs.close();
                                } catch (Exception e) {}
                        if (conn != null)
                                try {
                                        conn.close();
                                } catch (Exception e) {}
                }

                if(bannerList.size() > 0){
                        return UtilMisc.toMap("bannerList", bannerList);
                } else {
                        return UtilMisc.toMap("bannerList", FastList.newInstance());
                }
        }

Reply | Threaded
Open this post in threaded view
|

RE: Help Connecting to 2nd External Database

Jed Glazner
here are the errors from the stack trace.

java.lang.Thread.run(Thread.java:595)
java.lang.OutOfMemoryError: Java heap space
java.util.ArrayList.ensureCapacity(ArrayList.java:169)
java.util.ArrayList.add(ArrayList.java:351)
org.ofbiz.minerva.pool.ObjectPool.fillToMin(ObjectPool.java:961)
org.ofbiz.minerva.pool.ObjectPool.initialize(ObjectPool.java:533)
org.ofbiz.minerva.pool.jdbc.xa.XAPoolDataSource.initialize(XAPoolDataSource.java:313)
org.ofbiz.minerva.pool.jdbc.xa.XAPoolDataSource.getConnection(XAPoolDataSource.java:349)
org.ofbiz.entity.transaction.MinervaConnectionFactory.getConnection(MinervaConnectionFactory.java:115)
org.ofbiz.geronimo.GeronimoTransactionFactory.getConnection(GeronimoTransactionFactory.java:97)
org.ofbiz.entity.transaction.TransactionFactory.getConnection(TransactionFactory.java:98)
org.ofbiz.entity.jdbc.ConnectionFactory.getConnection(ConnectionFactory.java:75)
com.citydeals.tools.CityDealsServices.getBannerCampaignSummary(CityDealsServices.java:49)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
Reply | Threaded
Open this post in threaded view
|

RE: Help Connecting to 2nd External Database

SkipDever
In reply to this post by Jed Glazner
Whats the operating system and whats the "heap space" error?

Skip

-----Original Message-----
From: Jed Glazner [mailto:[hidden email]]
Sent: Monday, October 29, 2007 10:03 AM
To: [hidden email]
Cc: [hidden email]
Subject: Help Connecting to 2nd External Database


Hello,

I'm trying to connect to a 2nd external database (not opentaps) where I only have read only rights.  It's an adserver that powers the ads on our site.  
We only have read permissions and can't alter the tables at all.  our normal opentaps databse uses postgres, but the adserver uses MySQL.

What i've tried is to create a new datasource in the framework/entity/config/entityengine.xml.
I then wrote a service which uses the ConnectionFactory class.  The problem is that I get a Java Heap Space error the first time the service runs.
After that the service will run succesfully without any errors, until the next time ofbiz get started, then I get the heap space error once, and then it works fine again.  
I don't want to do just strait JDBC becauseI want to take advantage of the connection pooling within opentaps.

Any help would be appreciated.  The service and datasource are included.

-- datasource from entityengine.xml --

    <datasource name="mysql-advertpro"
            helper-class="org.ofbiz.entity.datasource.GenericHelperDAO"
            field-type-name="mysql"
            check-on-start="true"
            add-missing-on-start="false"
            check-pks-on-start="false"
            use-foreign-keys="true"
            join-style="ansi-no-parenthesis"
            alias-view-columns="false"
            drop-fk-use-foreign-key-keyword="true"
            table-type="MyISAM"
            character-set="latin1"
            collate="latin1_general_cs">
        <inline-jdbc
                jdbc-driver="com.mysql.jdbc.Driver"
                jdbc-uri="jdbc:mysql://<ipaddress>/advertpro?autoReconnect=true"
                jdbc-username="<username>"
                jdbc-password="<password>"
                isolation-level="ReadCommitted"
                pool-minsize="2"
                pool-maxsize="20"/>
    </datasource>

-- Custom Service --

public static Map getBannerCampaignDetails(DispatchContext ctx, Map context) {
                Connection conn = null;
                PreparedStatement statement = null;
                ResultSet rs = null;
                FastList bannerList = FastList.newInstance();
               
                String sql = "MySQL String Goes Here";
                               
                try{
                        conn = ConnectionFactory.getConnection("mysql-advertpro");
                        if (conn == null) {
                                throw new Exception("No advertpro connection configured");
                        }
                        statement = conn.prepareStatement(sql);
                        statement.setString(1, (String) context.get("partyId"));
                        rs = statement.executeQuery();
                        while(rs.next()){
                                FastMap row = FastMap.newInstance();
                                row.put("views", rs.getString("views"));
                                row.put("name", rs.getString("name"));
                                row.put("campaignId", rs.getString("campaignId"));
                                row.put("maxviews", rs.getString("maxviews"));
                                row.put("bannerType", rs.getString("bannerType"));
                                row.put("width", rs.getString("width"));
                                row.put("height", rs.getString("height"));
                                row.put("viewsRemaining", rs.getString("viewsRemaining"));
                                row.put("startdate", rs.getString("startdate"));
                                row.put("stopdate", rs.getString("stopdate"));
                                bannerList.add(row);
                        }
                        statement.close();
                        rs.close();
                } catch (Exception e){
                        ServiceUtil.returnError("Error getting connection to database" + e);
                }
               
                finally {
                        if (statement != null)
                                try {
                                        statement.close();
                                } catch (Exception e) {}
                        if (rs != null)
                                try {
                                        rs.close();
                                } catch (Exception e) {}
                        if (conn != null)
                                try {
                                        conn.close();
                                } catch (Exception e) {}
                }

                if(bannerList.size() > 0){
                        return UtilMisc.toMap("bannerList", bannerList);
                } else {
                        return UtilMisc.toMap("bannerList", FastList.newInstance());
                }
        }


Reply | Threaded
Open this post in threaded view
|

Re: Help Connecting to 2nd External Database

Jed Glazner
My Dev Box is running Kubuntu 7.10, but the Dev Server runs RHEL4.
I beleive the specific error is OutOfMemmory (java.lang.OutOfMemoryError: Java heap space)

Here is the full error from the log:

2007-10-29 09:14:10,789 (http-0.0.0.0-8080-Processor4) [  ServiceDispatcher.java:388:DEBUG] [[Sync service failed...- total:0.0,since last(Begin):0.0]] - 'crmsfa / getBannerCampaignSummary'
2007-10-29 09:14:10,791 (http-0.0.0.0-8080-Processor4) [  ServiceDispatcher.java:391:ERROR]
---- exception report ----------------------------------------------------------
Service [getBannerCampaignSummary] threw an unexpected exception/error
Exception: org.ofbiz.service.GenericServiceException
Message: Service target threw an unexpected exception (Java heap space)
---- stack trace ---------------------------------------------------------------
org.ofbiz.service.GenericServiceException: Service target threw an unexpected exception (Java heap space)
org.ofbiz.service.engine.StandardJavaEngine.serviceInvoker(StandardJavaEngine.java:106)
org.ofbiz.service.engine.StandardJavaEngine.runSync(StandardJavaEngine.java:56)
org.ofbiz.service.ServiceDispatcher.runSync(ServiceDispatcher.java:339)
org.ofbiz.service.ServiceDispatcher.runSync(ServiceDispatcher.java:214)
org.ofbiz.service.GenericDispatcher.runSync(GenericDispatcher.java:139)
sun.reflect.GeneratedMethodAccessor63.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:585)
bsh.Reflect.invokeMethod(Unknown Source)
bsh.Reflect.invokeObjectMethod(Unknown Source)
bsh.Name.invokeMethod(Unknown Source)
bsh.BSHMethodInvocation.eval(Unknown Source)
bsh.BSHPrimaryExpression.eval(Unknown Source)
bsh.BSHPrimaryExpression.eval(Unknown Source)
bsh.BSHVariableDeclarator.eval(Unknown Source)
bsh.BSHTypedVariableDeclaration.eval(Unknown Source)
bsh.Interpreter.evalParsedScript(Unknown Source)
bsh.Interpreter.evalParsedScript(Unknown Source)
org.ofbiz.base.util.BshUtil.runBshAtLocation(BshUtil.java:164)
org.ofbiz.widget.screen.ModelScreenAction$Script.runAction(ModelScreenAction.java:390)
org.ofbiz.widget.screen.ModelScreenAction.runSubActions(ModelScreenAction.java:120)
org.ofbiz.widget.screen.ModelScreenWidget$Section.renderWidgetString(ModelScreenWidget.java:241)
org.ofbiz.widget.screen.ModelScreen.renderScreenString(ModelScreen.java:137)
org.ofbiz.widget.screen.ScreenRenderer.render(ScreenRenderer.java:108)
org.ofbiz.widget.screen.ScreenRenderer.render(ScreenRenderer.java:93)
org.ofbiz.widget.screen.ScreenWidgetViewHandler.render(ScreenWidgetViewHandler.java:81)
org.ofbiz.webapp.control.RequestHandler.renderView(RequestHandler.java:645)
org.ofbiz.webapp.control.RequestHandler.doRequest(RequestHandler.java:429)
org.ofbiz.webapp.control.ControlServlet.doGet(ControlServlet.java:192)
javax.servlet.http.HttpServlet.service(HttpServlet.java:595)
javax.servlet.http.HttpServlet.service(HttpServlet.java:688)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:251)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:541)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
java.lang.Thread.run(Thread.java:595)
java.lang.OutOfMemoryError: Java heap space
java.util.ArrayList.ensureCapacity(ArrayList.java:169)
java.util.ArrayList.add(ArrayList.java:351)
org.ofbiz.minerva.pool.ObjectPool.fillToMin(ObjectPool.java:961)
org.ofbiz.minerva.pool.ObjectPool.initialize(ObjectPool.java:533)
org.ofbiz.minerva.pool.jdbc.xa.XAPoolDataSource.initialize(XAPoolDataSource.java:313)
org.ofbiz.minerva.pool.jdbc.xa.XAPoolDataSource.getConnection(XAPoolDataSource.java:349)
org.ofbiz.entity.transaction.MinervaConnectionFactory.getConnection(MinervaConnectionFactory.java:115)
org.ofbiz.geronimo.GeronimoTransactionFactory.getConnection(GeronimoTransactionFactory.java:97)
org.ofbiz.entity.transaction.TransactionFactory.getConnection(TransactionFactory.java:98)
org.ofbiz.entity.jdbc.ConnectionFactory.getConnection(ConnectionFactory.java:75)
com.citydeals.tools.CityDealsServices.getBannerCampaignSummary(CityDealsServices.java:49)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:585)
org.ofbiz.service.engine.StandardJavaEngine.serviceInvoker(StandardJavaEngine.java:94)
org.ofbiz.service.engine.StandardJavaEngine.runSync(StandardJavaEngine.java:56)
org.ofbiz.service.ServiceDispatcher.runSync(ServiceDispatcher.java:339)
org.ofbiz.service.ServiceDispatcher.runSync(ServiceDispatcher.java:214)
org.ofbiz.service.GenericDispatcher.runSync(GenericDispatcher.java:139)
sun.reflect.GeneratedMethodAccessor63.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:585)
bsh.Reflect.invokeMethod(Unknown Source)
bsh.Reflect.invokeObjectMethod(Unknown Source)
bsh.Name.invokeMethod(Unknown Source)
bsh.BSHMethodInvocation.eval(Unknown Source)
bsh.BSHPrimaryExpression.eval(Unknown Source)
bsh.BSHPrimaryExpression.eval(Unknown Source)
bsh.BSHVariableDeclarator.eval(Unknown Source)
bsh.BSHTypedVariableDeclaration.eval(Unknown Source)
bsh.Interpreter.evalParsedScript(Unknown Source)
--------------------------------------------------------------------------------


Jed.

On Monday 29 October 2007 11:29:34 am skip@theDevers wrote:

> Whats the operating system and whats the "heap space" error?
>
> Skip
>
> -----Original Message-----
> From: Jed Glazner [mailto:[hidden email]]
> Sent: Monday, October 29, 2007 10:03 AM
> To: [hidden email]
> Cc: [hidden email]
> Subject: Help Connecting to 2nd External Database
>
>
> Hello,
>
> I'm trying to connect to a 2nd external database (not opentaps) where I only have read only rights.  It's an adserver that powers the ads on our site.  
> We only have read permissions and can't alter the tables at all.  our normal opentaps databse uses postgres, but the adserver uses MySQL.
>
> What i've tried is to create a new datasource in the framework/entity/config/entityengine.xml.
> I then wrote a service which uses the ConnectionFactory class.  The problem is that I get a Java Heap Space error the first time the service runs.
> After that the service will run succesfully without any errors, until the next time ofbiz get started, then I get the heap space error once, and then it works fine again.  
> I don't want to do just strait JDBC becauseI want to take advantage of the connection pooling within opentaps.
>
> Any help would be appreciated.  The service and datasource are included.
>
> -- datasource from entityengine.xml --
>
>     <datasource name="mysql-advertpro"
>             helper-class="org.ofbiz.entity.datasource.GenericHelperDAO"
>             field-type-name="mysql"
>             check-on-start="true"
>             add-missing-on-start="false"
>             check-pks-on-start="false"
>             use-foreign-keys="true"
>             join-style="ansi-no-parenthesis"
>             alias-view-columns="false"
>             drop-fk-use-foreign-key-keyword="true"
>             table-type="MyISAM"
>             character-set="latin1"
>             collate="latin1_general_cs">
>         <inline-jdbc
>                 jdbc-driver="com.mysql.jdbc.Driver"
>                 jdbc-uri="jdbc:mysql://<ipaddress>/advertpro?autoReconnect=true"
>                 jdbc-username="<username>"
>                 jdbc-password="<password>"
>                 isolation-level="ReadCommitted"
>                 pool-minsize="2"
>                 pool-maxsize="20"/>
>     </datasource>
>
> -- Custom Service --
>
> public static Map getBannerCampaignDetails(DispatchContext ctx, Map context) {
> Connection conn = null;
> PreparedStatement statement = null;
> ResultSet rs = null;
> FastList bannerList = FastList.newInstance();
>
> String sql = "MySQL String Goes Here";
>
> try{
> conn = ConnectionFactory.getConnection("mysql-advertpro");
> if (conn == null) {
> throw new Exception("No advertpro connection configured");
> }
> statement = conn.prepareStatement(sql);
> statement.setString(1, (String) context.get("partyId"));
> rs = statement.executeQuery();
> while(rs.next()){
> FastMap row = FastMap.newInstance();
> row.put("views", rs.getString("views"));
> row.put("name", rs.getString("name"));
> row.put("campaignId", rs.getString("campaignId"));
> row.put("maxviews", rs.getString("maxviews"));
> row.put("bannerType", rs.getString("bannerType"));
> row.put("width", rs.getString("width"));
> row.put("height", rs.getString("height"));
> row.put("viewsRemaining", rs.getString("viewsRemaining"));
> row.put("startdate", rs.getString("startdate"));
> row.put("stopdate", rs.getString("stopdate"));
> bannerList.add(row);
> }
> statement.close();
> rs.close();
> } catch (Exception e){
> ServiceUtil.returnError("Error getting connection to database" + e);
> }
>
> finally {
> if (statement != null)
> try {
> statement.close();
> } catch (Exception e) {}
> if (rs != null)
> try {
> rs.close();
> } catch (Exception e) {}
> if (conn != null)
> try {
> conn.close();
> } catch (Exception e) {}
> }
>
> if(bannerList.size() > 0){
> return UtilMisc.toMap("bannerList", bannerList);
> } else {
> return UtilMisc.toMap("bannerList", FastList.newInstance());
> }
> }
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Help Connecting to 2nd External Database

Adrian Crum
In reply to this post by Jed Glazner
Jed Glazner wrote:
> I'm trying to connect to a 2nd external database (not opentaps) where I only have read only rights.  It's an adserver that powers the ads on our site.  
> We only have read permissions and can't alter the tables at all.  our normal opentaps databse uses postgres, but the adserver uses MySQL.
>
> What i've tried is to create a new datasource in the framework/entity/config/entityengine.xml.
> I then wrote a service which uses the ConnectionFactory class.  The problem is that I get a Java Heap Space error the first time the service runs.
> After that the service will run succesfully without any errors, until the next time ofbiz get started, then I get the heap space error once, and then it works fine again.  
> I don't want to do just strait JDBC becauseI want to take advantage of the connection pooling within opentaps.

Jed,

It looks to me like you're doing things the hard way. It would be simpler to use the Entity Engine
to gain access to the other database.

1. Create entity definitions for the external database like you would for an OFBiz entity (except
use the no-auto-stamp="true" attribute).
2. Assign the entities to a different group (in entitygroup.xml), "advertpro" for example.
3. Assign the "mysql-advertpro" datasource the the default delegator:
      <group-map group-name="advertpro" datasource-name="mysql-advertpro"/>
4. Use the delegator to access the external database.

-Adrian


Reply | Threaded
Open this post in threaded view
|

RE: Help Connecting to 2nd External Database

SkipDever
In reply to this post by Jed Glazner
Are you running the startofbiz script?  If so, you might try changing the MEMIF="-Xms128M -Xmx256M" to something more reasonable like MEMIF="-Xms512M -Xmx1024M" (assuming you have 2 gigs)

-----Original Message-----
From: Jed Glazner [mailto:[hidden email]]
Sent: Monday, October 29, 2007 10:38 AM
To: [hidden email]
Subject: Re: Help Connecting to 2nd External Database


My Dev Box is running Kubuntu 7.10, but the Dev Server runs RHEL4.
I beleive the specific error is OutOfMemmory (java.lang.OutOfMemoryError: Java heap space)

Here is the full error from the log:

2007-10-29 09:14:10,789 (http-0.0.0.0-8080-Processor4) [  ServiceDispatcher.java:388:DEBUG] [[Sync service failed...- total:0.0,since last(Begin):0.0]] - 'crmsfa / getBannerCampaignSummary'
2007-10-29 09:14:10,791 (http-0.0.0.0-8080-Processor4) [  ServiceDispatcher.java:391:ERROR]
---- exception report ----------------------------------------------------------
Service [getBannerCampaignSummary] threw an unexpected exception/error
Exception: org.ofbiz.service.GenericServiceException
Message: Service target threw an unexpected exception (Java heap space)
---- stack trace ---------------------------------------------------------------
org.ofbiz.service.GenericServiceException: Service target threw an unexpected exception (Java heap space)
org.ofbiz.service.engine.StandardJavaEngine.serviceInvoker(StandardJavaEngine.java:106)
org.ofbiz.service.engine.StandardJavaEngine.runSync(StandardJavaEngine.java:56)
org.ofbiz.service.ServiceDispatcher.runSync(ServiceDispatcher.java:339)
org.ofbiz.service.ServiceDispatcher.runSync(ServiceDispatcher.java:214)
org.ofbiz.service.GenericDispatcher.runSync(GenericDispatcher.java:139)
sun.reflect.GeneratedMethodAccessor63.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:585)
bsh.Reflect.invokeMethod(Unknown Source)
bsh.Reflect.invokeObjectMethod(Unknown Source)
bsh.Name.invokeMethod(Unknown Source)
bsh.BSHMethodInvocation.eval(Unknown Source)
bsh.BSHPrimaryExpression.eval(Unknown Source)
bsh.BSHPrimaryExpression.eval(Unknown Source)
bsh.BSHVariableDeclarator.eval(Unknown Source)
bsh.BSHTypedVariableDeclaration.eval(Unknown Source)
bsh.Interpreter.evalParsedScript(Unknown Source)
bsh.Interpreter.evalParsedScript(Unknown Source)
org.ofbiz.base.util.BshUtil.runBshAtLocation(BshUtil.java:164)
org.ofbiz.widget.screen.ModelScreenAction$Script.runAction(ModelScreenAction.java:390)
org.ofbiz.widget.screen.ModelScreenAction.runSubActions(ModelScreenAction.java:120)
org.ofbiz.widget.screen.ModelScreenWidget$Section.renderWidgetString(ModelScreenWidget.java:241)
org.ofbiz.widget.screen.ModelScreen.renderScreenString(ModelScreen.java:137)
org.ofbiz.widget.screen.ScreenRenderer.render(ScreenRenderer.java:108)
org.ofbiz.widget.screen.ScreenRenderer.render(ScreenRenderer.java:93)
org.ofbiz.widget.screen.ScreenWidgetViewHandler.render(ScreenWidgetViewHandler.java:81)
org.ofbiz.webapp.control.RequestHandler.renderView(RequestHandler.java:645)
org.ofbiz.webapp.control.RequestHandler.doRequest(RequestHandler.java:429)
org.ofbiz.webapp.control.ControlServlet.doGet(ControlServlet.java:192)
javax.servlet.http.HttpServlet.service(HttpServlet.java:595)
javax.servlet.http.HttpServlet.service(HttpServlet.java:688)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:251)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:541)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
java.lang.Thread.run(Thread.java:595)
java.lang.OutOfMemoryError: Java heap space
java.util.ArrayList.ensureCapacity(ArrayList.java:169)
java.util.ArrayList.add(ArrayList.java:351)
org.ofbiz.minerva.pool.ObjectPool.fillToMin(ObjectPool.java:961)
org.ofbiz.minerva.pool.ObjectPool.initialize(ObjectPool.java:533)
org.ofbiz.minerva.pool.jdbc.xa.XAPoolDataSource.initialize(XAPoolDataSource.java:313)
org.ofbiz.minerva.pool.jdbc.xa.XAPoolDataSource.getConnection(XAPoolDataSource.java:349)
org.ofbiz.entity.transaction.MinervaConnectionFactory.getConnection(MinervaConnectionFactory.java:115)
org.ofbiz.geronimo.GeronimoTransactionFactory.getConnection(GeronimoTransactionFactory.java:97)
org.ofbiz.entity.transaction.TransactionFactory.getConnection(TransactionFactory.java:98)
org.ofbiz.entity.jdbc.ConnectionFactory.getConnection(ConnectionFactory.java:75)
com.citydeals.tools.CityDealsServices.getBannerCampaignSummary(CityDealsServices.java:49)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:585)
org.ofbiz.service.engine.StandardJavaEngine.serviceInvoker(StandardJavaEngine.java:94)
org.ofbiz.service.engine.StandardJavaEngine.runSync(StandardJavaEngine.java:56)
org.ofbiz.service.ServiceDispatcher.runSync(ServiceDispatcher.java:339)
org.ofbiz.service.ServiceDispatcher.runSync(ServiceDispatcher.java:214)
org.ofbiz.service.GenericDispatcher.runSync(GenericDispatcher.java:139)
sun.reflect.GeneratedMethodAccessor63.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:585)
bsh.Reflect.invokeMethod(Unknown Source)
bsh.Reflect.invokeObjectMethod(Unknown Source)
bsh.Name.invokeMethod(Unknown Source)
bsh.BSHMethodInvocation.eval(Unknown Source)
bsh.BSHPrimaryExpression.eval(Unknown Source)
bsh.BSHPrimaryExpression.eval(Unknown Source)
bsh.BSHVariableDeclarator.eval(Unknown Source)
bsh.BSHTypedVariableDeclaration.eval(Unknown Source)
bsh.Interpreter.evalParsedScript(Unknown Source)
--------------------------------------------------------------------------------


Jed.

On Monday 29 October 2007 11:29:34 am skip@theDevers wrote:

> Whats the operating system and whats the "heap space" error?
>
> Skip
>
> -----Original Message-----
> From: Jed Glazner [mailto:[hidden email]]
> Sent: Monday, October 29, 2007 10:03 AM
> To: [hidden email]
> Cc: [hidden email]
> Subject: Help Connecting to 2nd External Database
>
>
> Hello,
>
> I'm trying to connect to a 2nd external database (not opentaps) where I only have read only rights.  It's an adserver that powers the ads on our site.  
> We only have read permissions and can't alter the tables at all.  our normal opentaps databse uses postgres, but the adserver uses MySQL.
>
> What i've tried is to create a new datasource in the framework/entity/config/entityengine.xml.
> I then wrote a service which uses the ConnectionFactory class.  The problem is that I get a Java Heap Space error the first time the service runs.
> After that the service will run succesfully without any errors, until the next time ofbiz get started, then I get the heap space error once, and then it works fine again.  
> I don't want to do just strait JDBC becauseI want to take advantage of the connection pooling within opentaps.
>
> Any help would be appreciated.  The service and datasource are included.
>
> -- datasource from entityengine.xml --
>
>     <datasource name="mysql-advertpro"
>             helper-class="org.ofbiz.entity.datasource.GenericHelperDAO"
>             field-type-name="mysql"
>             check-on-start="true"
>             add-missing-on-start="false"
>             check-pks-on-start="false"
>             use-foreign-keys="true"
>             join-style="ansi-no-parenthesis"
>             alias-view-columns="false"
>             drop-fk-use-foreign-key-keyword="true"
>             table-type="MyISAM"
>             character-set="latin1"
>             collate="latin1_general_cs">
>         <inline-jdbc
>                 jdbc-driver="com.mysql.jdbc.Driver"
>                 jdbc-uri="jdbc:mysql://<ipaddress>/advertpro?autoReconnect=true"
>                 jdbc-username="<username>"
>                 jdbc-password="<password>"
>                 isolation-level="ReadCommitted"
>                 pool-minsize="2"
>                 pool-maxsize="20"/>
>     </datasource>
>
> -- Custom Service --
>
> public static Map getBannerCampaignDetails(DispatchContext ctx, Map context) {
> Connection conn = null;
> PreparedStatement statement = null;
> ResultSet rs = null;
> FastList bannerList = FastList.newInstance();
>
> String sql = "MySQL String Goes Here";
>
> try{
> conn = ConnectionFactory.getConnection("mysql-advertpro");
> if (conn == null) {
> throw new Exception("No advertpro connection configured");
> }
> statement = conn.prepareStatement(sql);
> statement.setString(1, (String) context.get("partyId"));
> rs = statement.executeQuery();
> while(rs.next()){
> FastMap row = FastMap.newInstance();
> row.put("views", rs.getString("views"));
> row.put("name", rs.getString("name"));
> row.put("campaignId", rs.getString("campaignId"));
> row.put("maxviews", rs.getString("maxviews"));
> row.put("bannerType", rs.getString("bannerType"));
> row.put("width", rs.getString("width"));
> row.put("height", rs.getString("height"));
> row.put("viewsRemaining", rs.getString("viewsRemaining"));
> row.put("startdate", rs.getString("startdate"));
> row.put("stopdate", rs.getString("stopdate"));
> bannerList.add(row);
> }
> statement.close();
> rs.close();
> } catch (Exception e){
> ServiceUtil.returnError("Error getting connection to database" + e);
> }
>
> finally {
> if (statement != null)
> try {
> statement.close();
> } catch (Exception e) {}
> if (rs != null)
> try {
> rs.close();
> } catch (Exception e) {}
> if (conn != null)
> try {
> conn.close();
> } catch (Exception e) {}
> }
>
> if(bannerList.size() > 0){
> return UtilMisc.toMap("bannerList", bannerList);
> } else {
> return UtilMisc.toMap("bannerList", FastList.newInstance());
> }
> }
>
>
>



Reply | Threaded
Open this post in threaded view
|

Re: Help Connecting to 2nd External Database

BJ Freeman
In reply to this post by Jed Glazner
also note your running crmsfa.
you may want to check with them since that is where the error occured.
https://sourceforge.net/forum/forum.php?forum_id=559063

Jed Glazner sent the following on 10/29/2007 9:38 AM:

> My Dev Box is running Kubuntu 7.10, but the Dev Server runs RHEL4.
> I beleive the specific error is OutOfMemmory (java.lang.OutOfMemoryError: Java heap space)
>
> Here is the full error from the log:
>
> 2007-10-29 09:14:10,789 (http-0.0.0.0-8080-Processor4) [  ServiceDispatcher.java:388:DEBUG] [[Sync service failed...- total:0.0,since last(Begin):0.0]] - 'crmsfa / getBannerCampaignSummary'
> 2007-10-29 09:14:10,791 (http-0.0.0.0-8080-Processor4) [  ServiceDispatcher.java:391:ERROR]
> ---- exception report ----------------------------------------------------------
> Service [getBannerCampaignSummary] threw an unexpected exception/error
> Exception: org.ofbiz.service.GenericServiceException
> Message: Service target threw an unexpected exception (Java heap space)
> ---- stack trace ---------------------------------------------------------------
> org.ofbiz.service.GenericServiceException: Service target threw an unexpected exception (Java heap space)
> org.ofbiz.service.engine.StandardJavaEngine.serviceInvoker(StandardJavaEngine.java:106)
> org.ofbiz.service.engine.StandardJavaEngine.runSync(StandardJavaEngine.java:56)
> org.ofbiz.service.ServiceDispatcher.runSync(ServiceDispatcher.java:339)
> org.ofbiz.service.ServiceDispatcher.runSync(ServiceDispatcher.java:214)
> org.ofbiz.service.GenericDispatcher.runSync(GenericDispatcher.java:139)
> sun.reflect.GeneratedMethodAccessor63.invoke(Unknown Source)
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> java.lang.reflect.Method.invoke(Method.java:585)
> bsh.Reflect.invokeMethod(Unknown Source)
> bsh.Reflect.invokeObjectMethod(Unknown Source)
> bsh.Name.invokeMethod(Unknown Source)
> bsh.BSHMethodInvocation.eval(Unknown Source)
> bsh.BSHPrimaryExpression.eval(Unknown Source)
> bsh.BSHPrimaryExpression.eval(Unknown Source)
> bsh.BSHVariableDeclarator.eval(Unknown Source)
> bsh.BSHTypedVariableDeclaration.eval(Unknown Source)
> bsh.Interpreter.evalParsedScript(Unknown Source)
> bsh.Interpreter.evalParsedScript(Unknown Source)
> org.ofbiz.base.util.BshUtil.runBshAtLocation(BshUtil.java:164)
> org.ofbiz.widget.screen.ModelScreenAction$Script.runAction(ModelScreenAction.java:390)
> org.ofbiz.widget.screen.ModelScreenAction.runSubActions(ModelScreenAction.java:120)
> org.ofbiz.widget.screen.ModelScreenWidget$Section.renderWidgetString(ModelScreenWidget.java:241)
> org.ofbiz.widget.screen.ModelScreen.renderScreenString(ModelScreen.java:137)
> org.ofbiz.widget.screen.ScreenRenderer.render(ScreenRenderer.java:108)
> org.ofbiz.widget.screen.ScreenRenderer.render(ScreenRenderer.java:93)
> org.ofbiz.widget.screen.ScreenWidgetViewHandler.render(ScreenWidgetViewHandler.java:81)
> org.ofbiz.webapp.control.RequestHandler.renderView(RequestHandler.java:645)
> org.ofbiz.webapp.control.RequestHandler.doRequest(RequestHandler.java:429)
> org.ofbiz.webapp.control.ControlServlet.doGet(ControlServlet.java:192)
> javax.servlet.http.HttpServlet.service(HttpServlet.java:595)
> javax.servlet.http.HttpServlet.service(HttpServlet.java:688)
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
> org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:251)
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:541)
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
> java.lang.Thread.run(Thread.java:595)
> java.lang.OutOfMemoryError: Java heap space
> java.util.ArrayList.ensureCapacity(ArrayList.java:169)
> java.util.ArrayList.add(ArrayList.java:351)
> org.ofbiz.minerva.pool.ObjectPool.fillToMin(ObjectPool.java:961)
> org.ofbiz.minerva.pool.ObjectPool.initialize(ObjectPool.java:533)
> org.ofbiz.minerva.pool.jdbc.xa.XAPoolDataSource.initialize(XAPoolDataSource.java:313)
> org.ofbiz.minerva.pool.jdbc.xa.XAPoolDataSource.getConnection(XAPoolDataSource.java:349)
> org.ofbiz.entity.transaction.MinervaConnectionFactory.getConnection(MinervaConnectionFactory.java:115)
> org.ofbiz.geronimo.GeronimoTransactionFactory.getConnection(GeronimoTransactionFactory.java:97)
> org.ofbiz.entity.transaction.TransactionFactory.getConnection(TransactionFactory.java:98)
> org.ofbiz.entity.jdbc.ConnectionFactory.getConnection(ConnectionFactory.java:75)
> com.citydeals.tools.CityDealsServices.getBannerCampaignSummary(CityDealsServices.java:49)
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> java.lang.reflect.Method.invoke(Method.java:585)
> org.ofbiz.service.engine.StandardJavaEngine.serviceInvoker(StandardJavaEngine.java:94)
> org.ofbiz.service.engine.StandardJavaEngine.runSync(StandardJavaEngine.java:56)
> org.ofbiz.service.ServiceDispatcher.runSync(ServiceDispatcher.java:339)
> org.ofbiz.service.ServiceDispatcher.runSync(ServiceDispatcher.java:214)
> org.ofbiz.service.GenericDispatcher.runSync(GenericDispatcher.java:139)
> sun.reflect.GeneratedMethodAccessor63.invoke(Unknown Source)
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> java.lang.reflect.Method.invoke(Method.java:585)
> bsh.Reflect.invokeMethod(Unknown Source)
> bsh.Reflect.invokeObjectMethod(Unknown Source)
> bsh.Name.invokeMethod(Unknown Source)
> bsh.BSHMethodInvocation.eval(Unknown Source)
> bsh.BSHPrimaryExpression.eval(Unknown Source)
> bsh.BSHPrimaryExpression.eval(Unknown Source)
> bsh.BSHVariableDeclarator.eval(Unknown Source)
> bsh.BSHTypedVariableDeclaration.eval(Unknown Source)
> bsh.Interpreter.evalParsedScript(Unknown Source)
> --------------------------------------------------------------------------------
>
>
> Jed.
>
> On Monday 29 October 2007 11:29:34 am skip@theDevers wrote:
>> Whats the operating system and whats the "heap space" error?
>>
>> Skip
>>
>> -----Original Message-----
>> From: Jed Glazner [mailto:[hidden email]]
>> Sent: Monday, October 29, 2007 10:03 AM
>> To: [hidden email]
>> Cc: [hidden email]
>> Subject: Help Connecting to 2nd External Database
>>
>>
>> Hello,
>>
>> I'm trying to connect to a 2nd external database (not opentaps) where I only have read only rights.  It's an adserver that powers the ads on our site.  
>> We only have read permissions and can't alter the tables at all.  our normal opentaps databse uses postgres, but the adserver uses MySQL.
>>
>> What i've tried is to create a new datasource in the framework/entity/config/entityengine.xml.
>> I then wrote a service which uses the ConnectionFactory class.  The problem is that I get a Java Heap Space error the first time the service runs.
>> After that the service will run succesfully without any errors, until the next time ofbiz get started, then I get the heap space error once, and then it works fine again.  
>> I don't want to do just strait JDBC becauseI want to take advantage of the connection pooling within opentaps.
>>
>> Any help would be appreciated.  The service and datasource are included.
>>
>> -- datasource from entityengine.xml --
>>
>>     <datasource name="mysql-advertpro"
>>             helper-class="org.ofbiz.entity.datasource.GenericHelperDAO"
>>             field-type-name="mysql"
>>             check-on-start="true"
>>             add-missing-on-start="false"
>>             check-pks-on-start="false"
>>             use-foreign-keys="true"
>>             join-style="ansi-no-parenthesis"
>>             alias-view-columns="false"
>>             drop-fk-use-foreign-key-keyword="true"
>>             table-type="MyISAM"
>>             character-set="latin1"
>>             collate="latin1_general_cs">
>>         <inline-jdbc
>>                 jdbc-driver="com.mysql.jdbc.Driver"
>>                 jdbc-uri="jdbc:mysql://<ipaddress>/advertpro?autoReconnect=true"
>>                 jdbc-username="<username>"
>>                 jdbc-password="<password>"
>>                 isolation-level="ReadCommitted"
>>                 pool-minsize="2"
>>                 pool-maxsize="20"/>
>>     </datasource>
>>
>> -- Custom Service --
>>
>> public static Map getBannerCampaignDetails(DispatchContext ctx, Map context) {
>> Connection conn = null;
>> PreparedStatement statement = null;
>> ResultSet rs = null;
>> FastList bannerList = FastList.newInstance();
>>
>> String sql = "MySQL String Goes Here";
>>
>> try{
>> conn = ConnectionFactory.getConnection("mysql-advertpro");
>> if (conn == null) {
>> throw new Exception("No advertpro connection configured");
>> }
>> statement = conn.prepareStatement(sql);
>> statement.setString(1, (String) context.get("partyId"));
>> rs = statement.executeQuery();
>> while(rs.next()){
>> FastMap row = FastMap.newInstance();
>> row.put("views", rs.getString("views"));
>> row.put("name", rs.getString("name"));
>> row.put("campaignId", rs.getString("campaignId"));
>> row.put("maxviews", rs.getString("maxviews"));
>> row.put("bannerType", rs.getString("bannerType"));
>> row.put("width", rs.getString("width"));
>> row.put("height", rs.getString("height"));
>> row.put("viewsRemaining", rs.getString("viewsRemaining"));
>> row.put("startdate", rs.getString("startdate"));
>> row.put("stopdate", rs.getString("stopdate"));
>> bannerList.add(row);
>> }
>> statement.close();
>> rs.close();
>> } catch (Exception e){
>> ServiceUtil.returnError("Error getting connection to database" + e);
>> }
>>
>> finally {
>> if (statement != null)
>> try {
>> statement.close();
>> } catch (Exception e) {}
>> if (rs != null)
>> try {
>> rs.close();
>> } catch (Exception e) {}
>> if (conn != null)
>> try {
>> conn.close();
>> } catch (Exception e) {}
>> }
>>
>> if(bannerList.size() > 0){
>> return UtilMisc.toMap("bannerList", bannerList);
>> } else {
>> return UtilMisc.toMap("bannerList", FastList.newInstance());
>> }
>> }
>>
>>
>>
>
>
>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Help Connecting to 2nd External Database (resolution)

Jed Glazner
In reply to this post by Adrian Crum
Adrian,

Thank you for your help.  That resolved my problem.  After adding the datasource to the default delegator, and adding a new group and entity my code worked.

Jed.

On Monday 29 October 2007 11:43:59 am Adrian Crum wrote:

> Jed,
>
> It looks to me like you're doing things the hard way. It would be simpler to use the Entity Engine
> to gain access to the other database.
>
> 1. Create entity definitions for the external database like you would for an OFBiz entity (except
> use the no-auto-stamp="true" attribute).
> 2. Assign the entities to a different group (in entitygroup.xml), "advertpro" for example.
> 3. Assign the "mysql-advertpro" datasource the the default delegator:
>       <group-map group-name="advertpro" datasource-name="mysql-advertpro"/>
> 4. Use the delegator to access the external database.
>
> -Adrian