Administrator
|
Sorry Jinghai
With this commit a lot of tests don't pass Must be related with the httpmime-4.4.1.jar change or a dependency on solr component, I did not check Jacques Le 14/02/2016 12:59, [hidden email] a écrit : > Author: shijh > Date: Sun Feb 14 11:59:59 2016 > New Revision: 1730320 > > URL: http://svn.apache.org/viewvc?rev=1730320&view=rev > Log: > Refs OFBIZ-6715 Solr rebuild problem > > Added: > ofbiz/trunk/specialpurpose/solr/lib/runtime/httpmime-4.4.1.jar (with props) > Removed: > ofbiz/trunk/specialpurpose/solr/webapp/solr/WEB-INF/lib/httpmime-4.4.1.jar > Modified: > ofbiz/trunk/specialpurpose/solr/config/solrconfig.properties > ofbiz/trunk/specialpurpose/solr/home/solrdefault/conf/schema.xml > ofbiz/trunk/specialpurpose/solr/servicedef/solrservices.xml > ofbiz/trunk/specialpurpose/solr/src/org/ofbiz/solr/SolrProductSearch.java > ofbiz/trunk/specialpurpose/solr/src/org/ofbiz/solr/SolrUtil.java > > Modified: ofbiz/trunk/specialpurpose/solr/config/solrconfig.properties > URL: http://svn.apache.org/viewvc/ofbiz/trunk/specialpurpose/solr/config/solrconfig.properties?rev=1730320&r1=1730319&r2=1730320&view=diff > ============================================================================== > --- ofbiz/trunk/specialpurpose/solr/config/solrconfig.properties (original) > +++ ofbiz/trunk/specialpurpose/solr/config/solrconfig.properties Sun Feb 14 11:59:59 2016 > @@ -19,7 +19,7 @@ > # Solr configuration for custom OFBiz solr modules > > # Webapp access details > -solr.webapp.protocol=http > +solr.webapp.protocol=https > solr.webapp.domainName=localhost > # By default, port is same as OFBiz server, but can be overridden here. > solr.webapp.portOverride= > @@ -33,4 +33,15 @@ solr.eca.enabled=false > > # If true, connection errors during ECAs/SECAs are treated as warnings/failures rather than errors. > # If false, connection errors are treated as errors, and parent transactions are aborted. > -solr.eca.treatConnectErrorNonFatal=true > \ No newline at end of file > +solr.eca.treatConnectErrorNonFatal=true > + > +# The username and password of a solr client to CRUD an index > +solr.client.username=admin > +solr.client.password=ofbiz > + > +# Socket and connection timeout of a solr client > +solr.client.socket.timeout= > +solr.client.connection.timeout= > + > +# If true, trust self signed certification, default is false. > +solr.client.trust.selfsigned.cert=true > > Modified: ofbiz/trunk/specialpurpose/solr/home/solrdefault/conf/schema.xml > URL: http://svn.apache.org/viewvc/ofbiz/trunk/specialpurpose/solr/home/solrdefault/conf/schema.xml?rev=1730320&r1=1730319&r2=1730320&view=diff > ============================================================================== > --- ofbiz/trunk/specialpurpose/solr/home/solrdefault/conf/schema.xml (original) > +++ ofbiz/trunk/specialpurpose/solr/home/solrdefault/conf/schema.xml Sun Feb 14 11:59:59 2016 > @@ -45,7 +45,7 @@ > that avoids logging every request > --> > > -<schema name="example" version="1.5"> > +<schema name="products" version="1.5"> > <!-- attribute "name" is the name of this schema and is only used for display purposes. > version="x.y" is Solr's version number for the schema syntax and > semantics. It should not normally be changed by applications. > @@ -127,20 +127,27 @@ > in a compatible way. Any analysis applied to the <uniqueKey> should _not_ produce multiple > tokens > --> > - <field name="id" type="string" indexed="true" stored="true" required="true" multiValued="false" /> > + <!-- <field name="id" type="string" indexed="true" stored="true" required="true" multiValued="false" /> --> > > - <field name="sku" type="text_en_splitting_tight" indexed="true" stored="true" omitNorms="true"/> > - <field name="name" type="text_general" indexed="true" stored="true"/> > + <field name="productId" type="string" indexed="true" stored="true" required="true" multiValued="false" /> > + > + <field name="sku" type="text_general" indexed="true" stored="true"/> > + <field name="internalName" type="text_general" indexed="true" stored="true" multiValued="true"/> > + <field name="alphaNameSort" type="alphaOnlySort" indexed="true" stored="false"/> > <field name="manu" type="text_general" indexed="true" stored="true" omitNorms="true"/> > <field name="cat" type="string" indexed="true" stored="true" multiValued="true"/> > + <field name="catalog" type="string" indexed="true" stored="true" multiValued="true"/> > <field name="features" type="text_general" indexed="true" stored="true" multiValued="true"/> > - <field name="includes" type="text_general" indexed="true" stored="true" termVectors="true" termPositions="true" termOffsets="true" /> > - > + <field name="attributes" type="text_general" indexed="true" stored="true" multiValued="true"/> > <field name="weight" type="float" indexed="true" stored="true"/> > - <field name="price" type="float" indexed="true" stored="true"/> > + <field name="listPrice" type="float" indexed="true" stored="true"/> > + <field name="defaultPrice" type="float" indexed="true" stored="true"/> > <field name="popularity" type="int" indexed="true" stored="true" /> > - <field name="inStock" type="boolean" indexed="true" stored="true" /> > - > + <field name="inStock" type="int" indexed="true" stored="true" /> > + <field name="isVirtual" type="boolean" indexed="true" stored="true" /> > + <field name="isDigital" type="boolean" indexed="true" stored="true" /> > + <field name="isPhysical" type="boolean" indexed="true" stored="true" /> > + > <field name="store" type="location" indexed="true" stored="true"/> > > <!-- Common metadata fields, named specifically to match up with > @@ -152,18 +159,27 @@ > "resourcename": From SolrCell request param resource.name > --> > <field name="title" type="text_general" indexed="true" stored="true" multiValued="true"/> > + <field name="title_i18n_en" type="text_general" indexed="true" stored="true" multiValued="true"/> > + <field name="title_i18n_de" type="text_general" indexed="true" stored="true" multiValued="true"/> > + <field name="title_i18n_fr" type="text_general" indexed="true" stored="true" multiValued="true"/> > <field name="subject" type="text_general" indexed="true" stored="true"/> > <field name="description" type="text_general" indexed="true" stored="true"/> > + <field name="description_i18n_en" type="text_general" indexed="true" stored="true"/> > + <field name="description_i18n_de" type="text_general" indexed="true" stored="true"/> > + <field name="description_i18n_fr" type="text_general" indexed="true" stored="true"/> > + <field name="longdescription" type="text_general" indexed="true" stored="true"/> > + <field name="longdescription_i18n_en" type="text_general" indexed="true" stored="true"/> > + <field name="longdescription_i18n_de" type="text_general" indexed="true" stored="true"/> > + <field name="longdescription_i18n_fr" type="text_general" indexed="true" stored="true"/> > <field name="comments" type="text_general" indexed="true" stored="true"/> > <field name="author" type="text_general" indexed="true" stored="true"/> > <field name="keywords" type="text_general" indexed="true" stored="true"/> > - <field name="category" type="text_general" indexed="true" stored="true"/> > - <field name="resourcename" type="text_general" indexed="true" stored="true"/> > - <field name="url" type="text_general" indexed="true" stored="true"/> > <field name="content_type" type="string" indexed="true" stored="true" multiValued="true"/> > <field name="last_modified" type="date" indexed="true" stored="true"/> > <field name="links" type="string" indexed="true" stored="true" multiValued="true"/> > - <field name="_src_" type="string" indexed="false" stored="true"/> > + <field name="smallImage" type="text_general" indexed="true" stored="true"/> > + <field name="mediumImage" type="text_general" indexed="true" stored="true"/> > + <field name="largeImage" type="text_general" indexed="true" stored="true"/> > > <!-- Main body of document extracted by SolrCell. > NOTE: This field is not indexed by default, since it is also copied to "text" > @@ -255,7 +271,7 @@ > <!-- Field to use to determine and enforce document uniqueness. > Unless this field is marked with required="false", it will be a required field > --> > - <uniqueKey>id</uniqueKey> > + <uniqueKey>productId</uniqueKey> > > <!-- DEPRECATED: The defaultSearchField is consulted by various query parsers when > parsing a query string that isn't explicit about the field. Machine (non-user) > @@ -277,28 +293,15 @@ > or to add multiple fields to the same field for easier/faster searching. --> > > <copyField source="cat" dest="text"/> > - <copyField source="name" dest="text"/> > + <copyField source="internalName" dest="text"/> > <copyField source="manu" dest="text"/> > <copyField source="features" dest="text"/> > - <copyField source="includes" dest="text"/> > + <copyField source="attributes" dest="text"/> > + <copyField source="*_i18n_en" dest="text" /> > + <copyField source="*_i18n_de" dest="text" /> > + <copyField source="*_i18n_fr" dest="text" /> > <copyField source="manu" dest="manu_exact"/> > > - <!-- Copy the price into a currency enabled field (default USD) --> > - <copyField source="price" dest="price_c"/> > - > - <!-- Text fields from SolrCell to search by default in our catch-all field --> > - <copyField source="title" dest="text"/> > - <copyField source="author" dest="text"/> > - <copyField source="description" dest="text"/> > - <copyField source="keywords" dest="text"/> > - <copyField source="content" dest="text"/> > - <copyField source="content_type" dest="text"/> > - <copyField source="resourcename" dest="text"/> > - <copyField source="url" dest="text"/> > - > - <!-- Create a string version of author for faceting --> > - <copyField source="author" dest="author_s"/> > - > <!-- Above, multiple source fields are copied to the [text] field. > Another way to map multiple source fields to the same > destination field is to use the dynamic field syntax. > > Added: ofbiz/trunk/specialpurpose/solr/lib/runtime/httpmime-4.4.1.jar > URL: http://svn.apache.org/viewvc/ofbiz/trunk/specialpurpose/solr/lib/runtime/httpmime-4.4.1.jar?rev=1730320&view=auto > ============================================================================== > Binary file - no diff available. > > Propchange: ofbiz/trunk/specialpurpose/solr/lib/runtime/httpmime-4.4.1.jar > ------------------------------------------------------------------------------ > svn:mime-type = application/octet-stream > > Modified: ofbiz/trunk/specialpurpose/solr/servicedef/solrservices.xml > URL: http://svn.apache.org/viewvc/ofbiz/trunk/specialpurpose/solr/servicedef/solrservices.xml?rev=1730320&r1=1730319&r2=1730320&view=diff > ============================================================================== > --- ofbiz/trunk/specialpurpose/solr/servicedef/solrservices.xml (original) > +++ ofbiz/trunk/specialpurpose/solr/servicedef/solrservices.xml Sun Feb 14 11:59:59 2016 > @@ -24,20 +24,13 @@ under the License. > <description>Content Component Services</description> > <vendor>OFBiz</vendor> > > - <!-- Test Case --> > - <service name="SolrProductSearch" engine="java" > - transaction-timeout="72000" > - location="org.ofbiz.solr.SolrProductSearch" > - invoke="SolrProductSearch" debug="true" validate="true"> > - <description>First Test with Solr</description> > - </service> > - > <!-- Rebuild the Solr Tree --> > <service name="rebuildSolrIndex" engine="java" > transaction-timeout="72000" > location="org.ofbiz.solr.SolrProductSearch" > invoke="rebuildSolrIndex" debug="true" validate="true"> > <description>rebuild SOLR Index</description> > + <attribute mode="IN" name="indexName" optional="false" type="String"/> > <attribute mode="IN" name="treatConnectErrorNonFatal" optional="true" type="Boolean" default-value="false" /> > </service> > > @@ -48,6 +41,7 @@ under the License. > invoke="addToSolr" debug="true" validate="true"> > <description>Adds product to solr, with product denoted by productId field in instance attribute > - intended for use with ECAs/SECAs</description> > + <attribute mode="IN" name="indexName" optional="false" type="String"/> > <attribute mode="IN" name="instance" optional="false" type="org.ofbiz.entity.GenericValue" /> > </service> > > @@ -56,6 +50,7 @@ under the License. > location="org.ofbiz.solr.SolrProductSearch" > invoke="addToSolrIndex" debug="true" validate="true"> > <description>Add a Product to Solr Index</description> > + <attribute mode="IN" name="indexName" optional="false" type="String"/> > <attribute mode="IN" name="treatConnectErrorNonFatal" optional="true" type="Boolean" /> > <attribute mode="IN" name="productId" optional="false" type="String" /> > <attribute mode="IN" name="sku" optional="true" type="String" /> > @@ -88,6 +83,7 @@ under the License. > location="org.ofbiz.solr.SolrProductSearch" > invoke="addListToSolrIndex" debug="true" validate="true"> > <description>Add a List of Products to Solr Index and flush after all have been added</description> > + <attribute mode="IN" name="indexName" optional="false" type="String"/> > <attribute mode="IN" name="treatConnectErrorNonFatal" optional="true" type="Boolean" /> > <attribute mode="IN" name="fieldList" optional="false" type="List" /> > <attribute mode="OUT" name="errorType" optional="true" type="String" /> > @@ -99,6 +95,7 @@ under the License. > location="org.ofbiz.solr.SolrProductSearch" > invoke="productsSearch" debug="true" validate="true"> > <description>Run a query on Solr and return the results</description> > + <attribute mode="IN" name="indexName" optional="false" type="String"/> > <attribute mode="IN" optional="false" name="productCategoryId" type="String"/> > <attribute mode="IN" optional="true" name="viewSize" type="String"/> > <attribute mode="IN" optional="true" name="viewIndex" type="String"/> > @@ -115,6 +112,7 @@ under the License. > location="org.ofbiz.solr.SolrProductSearch" > invoke="keywordSearch" debug="true" validate="true"> > <description>Run a query on Solr and return the results</description> > + <attribute mode="IN" name="indexName" optional="false" type="String"/> > <attribute mode="IN" optional="false" name="query" type="String"/> > <attribute mode="IN" optional="true" name="viewSize" type="String"/> > <attribute mode="IN" optional="true" name="viewIndex" type="String"/> > @@ -139,6 +137,7 @@ under the License. > location="org.ofbiz.solr.SolrProductSearch" > invoke="runSolrQuery" debug="true" validate="true"> > <description>Run a query on Solr and return the results</description> > + <attribute mode="IN" name="indexName" optional="false" type="String"/> > <attribute mode="IN" optional="false" name="query" type="String"/> > <attribute mode="IN" optional="true" name="viewSize" type="Integer"/> > <attribute mode="IN" optional="true" name="viewIndex" type="Integer"/> > @@ -158,6 +157,7 @@ under the License. > location="org.ofbiz.solr.SolrProductSearch" > invoke="getAvailableCategories" debug="true" validate="true"> > <description>Run a query on Solr and return the results</description> > + <attribute mode="IN" name="indexName" optional="false" type="String"/> > <attribute mode="IN" optional="true" name="productCategoryId" type="String"/> > <attribute mode="IN" optional="true" name="productId" type="String"/> > <attribute mode="IN" optional="true" name="catalogId" type="String"/> > @@ -173,6 +173,7 @@ under the License. > location="org.ofbiz.solr.SolrProductSearch" > invoke="getSideDeepCategories" debug="true" validate="true"> > <description>Run a query on Solr and return the results</description> > + <attribute mode="IN" name="indexName" optional="false" type="String"/> > <attribute mode="IN" optional="true" name="productCategoryId" type="String"/> > <attribute mode="IN" optional="true" name="catalogId" type="String"/> > <attribute name="numFound" type="Long" mode="OUT" optional="false"/> > > Modified: ofbiz/trunk/specialpurpose/solr/src/org/ofbiz/solr/SolrProductSearch.java > URL: http://svn.apache.org/viewvc/ofbiz/trunk/specialpurpose/solr/src/org/ofbiz/solr/SolrProductSearch.java?rev=1730320&r1=1730319&r2=1730320&view=diff > ============================================================================== > --- ofbiz/trunk/specialpurpose/solr/src/org/ofbiz/solr/SolrProductSearch.java (original) > +++ ofbiz/trunk/specialpurpose/solr/src/org/ofbiz/solr/SolrProductSearch.java Sun Feb 14 11:59:59 2016 > @@ -62,7 +62,6 @@ public abstract class SolrProductSearch > > public static final String module = SolrProductSearch.class.getName(); > > - > /** > * Adds product to solr, with product denoted by productId field in instance attribute > * - intended for use with ECAs/SECAs. > @@ -73,6 +72,7 @@ public abstract class SolrProductSearch > Delegator delegator = dctx.getDelegator(); > GenericValue productInstance = (GenericValue) context.get("instance"); > String productId = (String) productInstance.get("productId"); > + String solrIndexName = (String) context.get("indexName"); > > if (SolrUtil.isSolrEcaEnabled()) { > // Debug.logVerbose("Solr: addToSolr: Running indexing for productId '" + productId + "'", module); > @@ -80,6 +80,7 @@ public abstract class SolrProductSearch > GenericValue product = delegator.findOne("Product", UtilMisc.toMap("productId", productId), false); > Map<String, Object> dispatchContext = ProductUtil.getProductContent(product, dctx, context); > dispatchContext.put("treatConnectErrorNonFatal", SolrUtil.isEcaTreatConnectErrorNonFatal()); > + dispatchContext.put("indexName", solrIndexName); > Map<String, Object> runResult = dispatcher.runSync("addToSolrIndex", dispatchContext); > String runMsg = ServiceUtil.getErrorMessage(runResult); > if (UtilValidate.isEmpty(runMsg)) { > @@ -114,13 +115,14 @@ public abstract class SolrProductSearch > HttpSolrClient client = null; > Map<String, Object> result; > String productId = (String) context.get("productId"); > + String solrIndexName = (String) context.get("indexName"); > // connectErrorNonFatal is a necessary option because in some cases it may be considered normal that solr server is unavailable; > // don't want to return error and abort transactions in these cases. > Boolean treatConnectErrorNonFatal = (Boolean) context.get("treatConnectErrorNonFatal"); > try { > Debug.logInfo("Solr: Generating and indexing document for productId '" + productId + "'", module); > > - client = new HttpSolrClient(SolrUtil.solrUrl); > + client = SolrUtil.getInstance().getHttpSolrClient(solrIndexName); > //Debug.log(server.ping().toString()); > > // Construct Documents > @@ -185,6 +187,7 @@ public abstract class SolrProductSearch > * This is faster than reflushing the index each time. > */ > public static Map<String, Object> addListToSolrIndex(DispatchContext dctx, Map<String, Object> context) throws GenericEntityException { > + String solrIndexName = (String) context.get("indexName"); > HttpSolrClient client = null; > Map<String, Object> result; > Boolean treatConnectErrorNonFatal = (Boolean) context.get("treatConnectErrorNonFatal"); > @@ -204,7 +207,7 @@ public abstract class SolrProductSearch > docs.add(doc1); > } > // push Documents to server > - client = new HttpSolrClient(SolrUtil.solrUrl); > + client = SolrUtil.getInstance().getHttpSolrClient(solrIndexName); > client.add(docs); > client.commit(); > > @@ -258,9 +261,10 @@ public abstract class SolrProductSearch > public static Map<String, Object> runSolrQuery(DispatchContext dctx, Map<String, Object> context) { > // get Connection > HttpSolrClient client = null; > + String solrIndexName = (String) context.get("indexName"); > Map<String, Object> result; > try { > - client = new HttpSolrClient(SolrUtil.solrUrl); > + client = SolrUtil.getInstance().getHttpSolrClient(solrIndexName); > // create Query Object > SolrQuery solrQuery = new SolrQuery(); > solrQuery.setQuery((String) context.get("query")); > @@ -363,6 +367,7 @@ public abstract class SolrProductSearch > public static Map<String, Object> productsSearch(DispatchContext dctx, Map<String, Object> context) { > Map<String, Object> result; > LocalDispatcher dispatcher = dctx.getDispatcher(); > + String solrIndexName = (String) context.get("indexName"); > > try { > Map<String, Object> dispatchMap = new HashMap<String, Object>(); > @@ -381,6 +386,7 @@ public abstract class SolrProductSearch > dispatchMap.put("facet", false); > dispatchMap.put("spellcheck", true); > dispatchMap.put("highlight", true); > + dispatchMap.put("indexName", solrIndexName); > Map<String, Object> searchResult = dispatcher.runSync("runSolrQuery", dispatchMap); > QueryResponse queryResult = (QueryResponse) searchResult.get("queryResult"); > result = ServiceUtil.returnSuccess(); > @@ -403,6 +409,7 @@ public abstract class SolrProductSearch > public static Map<String, Object> keywordSearch(DispatchContext dctx, Map<String, Object> context) { > Map<String, Object> result; > LocalDispatcher dispatcher = dctx.getDispatcher(); > + String solrIndexName = (String) context.get("indexName"); > > try { > if (context.get("query") == null || context.get("query").equals("")) > @@ -418,6 +425,8 @@ public abstract class SolrProductSearch > if (context.get("queryFilter") != null) > dispatchMap.put("queryFilter", context.get("queryFilter")); > dispatchMap.put("spellcheck", true); > + dispatchMap.put("indexName", solrIndexName); > + > Map<String, Object> searchResult = dispatcher.runSync("runSolrQuery", dispatchMap); > QueryResponse queryResult = (QueryResponse) searchResult.get("queryResult"); > > @@ -479,6 +488,7 @@ public abstract class SolrProductSearch > */ > public static Map<String, Object> getAvailableCategories(DispatchContext dctx, Map<String, Object> context) { > Map<String, Object> result; > + String solrIndexName = (String) context.get("indexName"); > try { > boolean displayProducts = false; > if (UtilValidate.isNotEmpty(context.get("displayProducts"))) > @@ -494,10 +504,9 @@ public abstract class SolrProductSearch > if (UtilValidate.isNotEmpty(context.get("catalogId"))) > catalogId = (String) context.get("catalogId"); > > - //String productCategoryId = (String) context.get("productCategoryId") != null ? CategoryUtil.getCategoryNameWithTrail((String) context.get("productCategoryId"), dctx): null; > String productCategoryId = (String) context.get("productCategoryId") != null ? CategoryUtil.getCategoryNameWithTrail((String) context.get("productCategoryId"),dctx) : null; > - Debug.logInfo("productCategoryId "+productCategoryId, module); > - Map<String, Object> query = SolrUtil.categoriesAvailable(catalogId, productCategoryId, (String) context.get("productId"), displayProducts, viewIndex, viewSize); > + Debug.logInfo("productCategoryId " + productCategoryId, module); > + Map<String, Object> query = SolrUtil.categoriesAvailable(catalogId, productCategoryId, (String) context.get("productId"), displayProducts, viewIndex, viewSize, solrIndexName); > > QueryResponse cat = (QueryResponse) query.get("rows"); > result = ServiceUtil.returnSuccess(); > @@ -508,7 +517,7 @@ public abstract class SolrProductSearch > FacetField field = (FacetField) catIterator.next(); > List<Count> catL = (List<Count>) field.getValues(); > if (catL != null) { > - // log.info("FacetFields = "+catL); > + // Debug.logInfo("FacetFields = " + catL, module); > for (Iterator<Count> catIter = catL.iterator(); catIter.hasNext();) { > FacetField.Count f = (FacetField.Count) catIter.next(); > if (f.getCount() > 0) { > @@ -517,7 +526,7 @@ public abstract class SolrProductSearch > } > result.put("categories", categories); > result.put("numFound", cat.getResults().getNumFound()); > - // log.info("The returned map is this:"+result); > + // Debug.logInfo("The returned map is this:" + result, module); > } > } > } catch (Exception e) { > @@ -533,6 +542,7 @@ public abstract class SolrProductSearch > */ > public static Map<String, Object> getSideDeepCategories(DispatchContext dctx, Map<String, Object> context) { > Map<String, Object> result; > + String solrIndexName = (String) context.get("indexName"); > try { > String catalogId = null; > if (UtilValidate.isNotEmpty(context.get("catalogId"))) > @@ -556,7 +566,7 @@ public abstract class SolrProductSearch > int level = Integer.parseInt(categoryPathArray[0]); > String facetQuery = CategoryUtil.getFacetFilterForCategory(categoryPath, dctx); > //Debug.logInfo("categoryPath: "+categoryPath + " facetQuery: "+facetQuery,module); > - Map<String, Object> query = SolrUtil.categoriesAvailable(catalogId, categoryPath, null, facetQuery,false, 0, 0); > + Map<String, Object> query = SolrUtil.categoriesAvailable(catalogId, categoryPath, null, facetQuery, false, 0, 0, solrIndexName); > QueryResponse cat = (QueryResponse) query.get("rows"); > List<Map<String, Object>> categories = new ArrayList<Map<String, Object>>(); > > @@ -612,12 +622,13 @@ public abstract class SolrProductSearch > GenericDelegator delegator = (GenericDelegator) dctx.getDelegator(); > LocalDispatcher dispatcher = dctx.getDispatcher(); > GenericValue userLogin = (GenericValue) context.get("userLogin"); > - Locale locale = new Locale("de_DE"); > + Locale locale = (Locale) context.get("locale"); > + String solrIndexName = (String) context.get("indexName"); > > Boolean treatConnectErrorNonFatal = (Boolean) context.get("treatConnectErrorNonFatal"); > > try { > - client = new HttpSolrClient(SolrUtil.solrUrl); > + client = SolrUtil.getInstance().getHttpSolrClient(solrIndexName); > > // now lets fetch all products > List<Map<String, Object>> solrDocs = new ArrayList<Map<String, Object>>(); > @@ -642,7 +653,7 @@ public abstract class SolrProductSearch > > // THis adds all products to the Index (instantly) > Map<String, Object> runResult = dispatcher.runSync("addListToSolrIndex", UtilMisc.toMap("fieldList", solrDocs, "userLogin", userLogin, > - "locale", locale, "treatConnectErrorNonFatal", treatConnectErrorNonFatal)); > + "locale", locale, "indexName", solrIndexName, "treatConnectErrorNonFatal", treatConnectErrorNonFatal)); > > String runMsg = ServiceUtil.getErrorMessage(runResult); > if (UtilValidate.isEmpty(runMsg)) { > > Modified: ofbiz/trunk/specialpurpose/solr/src/org/ofbiz/solr/SolrUtil.java > URL: http://svn.apache.org/viewvc/ofbiz/trunk/specialpurpose/solr/src/org/ofbiz/solr/SolrUtil.java?rev=1730320&r1=1730319&r2=1730320&view=diff > ============================================================================== > --- ofbiz/trunk/specialpurpose/solr/src/org/ofbiz/solr/SolrUtil.java (original) > +++ ofbiz/trunk/specialpurpose/solr/src/org/ofbiz/solr/SolrUtil.java Sun Feb 14 11:59:59 2016 > @@ -18,12 +18,26 @@ > *******************************************************************************/ > package org.ofbiz.solr; > > +import java.io.IOException; > import java.util.HashMap; > import java.util.Iterator; > import java.util.List; > import java.util.Map; > import java.util.Set; > > +import javax.net.ssl.SSLContext; > + > +import org.apache.http.client.ClientProtocolException; > +import org.apache.http.client.config.RequestConfig; > +import org.apache.http.client.methods.CloseableHttpResponse; > +import org.apache.http.client.methods.HttpGet; > +import org.apache.http.client.protocol.HttpClientContext; > +import org.apache.http.conn.ssl.NoopHostnameVerifier; > +import org.apache.http.conn.ssl.SSLConnectionSocketFactory; > +import org.apache.http.ssl.SSLContexts; > +import org.apache.http.conn.ssl.TrustSelfSignedStrategy; > +import org.apache.http.impl.client.CloseableHttpClient; > +import org.apache.http.impl.client.HttpClients; > import org.apache.solr.client.solrj.SolrQuery; > import org.apache.solr.client.solrj.SolrRequest.METHOD; > import org.apache.solr.client.solrj.impl.HttpSolrClient; > @@ -33,6 +47,7 @@ import org.ofbiz.base.component.Componen > import org.ofbiz.base.component.ComponentConfig.WebappInfo; > import org.ofbiz.base.component.ComponentException; > import org.ofbiz.base.util.Debug; > +import org.ofbiz.base.util.FileUtil; > import org.ofbiz.base.util.UtilGenerics; > import org.ofbiz.base.util.UtilProperties; > import org.ofbiz.base.util.UtilValidate; > @@ -41,7 +56,7 @@ import org.ofbiz.entity.GenericEntityExc > /** > * Solr utility class. > */ > -public abstract class SolrUtil { > +public final class SolrUtil { > > public static final String module = SolrUtil.class.getName(); > private static String[] solrProdAttribute = { "productId", "internalName", "manu", "size", "smallImage", "mediumImage", "largeImage", "listPrice", "defaultPrice", "inStock", "isVirtual" }; > @@ -49,7 +64,27 @@ public abstract class SolrUtil { > public static final String solrConfigName = "solrconfig.properties"; > public static final String solrUrl = makeSolrWebappUrl(); > > - public static String makeSolrWebappUrl() { > + protected static final String socketTimeoutString = UtilProperties.getPropertyValue(solrConfigName, "solr.client.socket.timeout"); > + > + protected static final String connectionTimeoutString = UtilProperties.getPropertyValue(solrConfigName, "solr.client.connection.timeout"); > + > + protected static final String clientUsername = UtilProperties.getPropertyValue(solrConfigName, "solr.client.username"); > + > + protected static final String clientPassword = UtilProperties.getPropertyValue(solrConfigName, "solr.client.password"); > + > + protected static final Integer socketTimeout = getSocketTimeout(); > + > + protected static final Integer connectionTimeout = getConnectionTimeout(); > + > + protected static final String trustSelfSignedCertString = UtilProperties.getPropertyValue(solrConfigName, "solr.client.trust.selfsigned.cert", "false"); > + > + protected static final boolean trustSelfSignedCert = getTrustSelfSignedCert(); > + > + protected SolrUtil() { > + // empty constructor > + } > + > + public static String makeSolrWebappUrl() { > final String solrWebappProtocol = UtilProperties.getPropertyValue(solrConfigName, "solr.webapp.protocol"); > final String solrWebappDomainName = UtilProperties.getPropertyValue(solrConfigName, "solr.webapp.domainName"); > final String solrWebappPath = UtilProperties.getPropertyValue(solrConfigName, "solr.webapp.path"); > @@ -58,14 +93,42 @@ public abstract class SolrUtil { > String solrPort; > if (UtilValidate.isNotEmpty(solrWebappPortOverride)) { > solrPort = solrWebappPortOverride; > - } > - else { > + } else { > solrPort = UtilProperties.getPropertyValue("url", ("https".equals(solrWebappProtocol) ? "port.https" : "port.http")); > } > > return solrWebappProtocol + "://" + solrWebappDomainName + ":" + solrPort + solrWebappPath; > } > > + private static Integer getSocketTimeout() { > + if (UtilValidate.isNotEmpty(socketTimeoutString)) { > + try { > + return Integer.parseInt(socketTimeoutString); > + } catch (Exception e) { > + return null; > + } > + } > + return null; > + } > + > + private static Integer getConnectionTimeout() { > + if (UtilValidate.isNotEmpty(connectionTimeoutString)) { > + try { > + return Integer.parseInt(connectionTimeoutString); > + } catch (Exception e) { > + return null; > + } > + } > + return null; > + } > + > + private static boolean getTrustSelfSignedCert() { > + if ("true".equals(trustSelfSignedCertString)) { > + return true; > + } > + return false; > + } > + > public static boolean isSolrEcaEnabled() { > Boolean ecaEnabled = null; > String sysProp = System.getProperty("ofbiz.solr.eca.enabled"); > @@ -181,18 +244,18 @@ public abstract class SolrUtil { > return doc1; > } > > - public static Map<String, Object> categoriesAvailable(String catalogId, String categoryId, String productId, boolean displayproducts, int viewIndex, int viewSize) { > - return categoriesAvailable(catalogId,categoryId,productId,null,displayproducts,viewIndex,viewSize); > + public static Map<String, Object> categoriesAvailable(String catalogId, String categoryId, String productId, boolean displayproducts, int viewIndex, int viewSize, String solrIndexName) { > + return categoriesAvailable(catalogId, categoryId, productId, null, displayproducts, viewIndex, viewSize, solrIndexName); > } > > - public static Map<String, Object> categoriesAvailable(String catalogId, String categoryId, String productId, String facetPrefix, boolean displayproducts, int viewIndex, int viewSize) { > + public static Map<String, Object> categoriesAvailable(String catalogId, String categoryId, String productId, String facetPrefix, boolean displayproducts, int viewIndex, int viewSize, String solrIndexName) { > // create the data model > Map<String, Object> result = new HashMap<String, Object>(); > HttpSolrClient client = null; > QueryResponse returnMap = new QueryResponse(); > try { > // do the basic query > - client = new HttpSolrClient(solrUrl); > + client = getInstance().getHttpSolrClient(solrIndexName); > // create Query Object > String query = "inStock[1 TO *]"; > if (categoryId != null) > @@ -235,4 +298,68 @@ public abstract class SolrUtil { > return result; > } > > + private CloseableHttpClient getAllowAllHttpClient() { > + try { > + // Trust own CA and all self-signed certs > + SSLContext sslContext = SSLContexts.custom() > + .loadTrustMaterial(FileUtil.getFile("component://base/config/ofbizssl.jks"), "changeit".toCharArray(), > + new TrustSelfSignedStrategy()) > + .build(); > + // No host name verifier > + SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory( > + sslContext, > + NoopHostnameVerifier.INSTANCE); > + CloseableHttpClient httpClient = HttpClients.custom() > + .setSSLSocketFactory(sslsf) > + .build(); > + return httpClient; > + } catch (Exception e) { > + return HttpClients.createDefault(); > + } > + } > + > + public static SolrUtil getInstance() { > + return new SolrUtil(); > + } > + > + public HttpSolrClient getHttpSolrClient(String solrIndexName) throws ClientProtocolException, IOException { > + HttpClientContext httpContext = HttpClientContext.create(); > + > + CloseableHttpClient httpClient = null; > + if (trustSelfSignedCert) { > + httpClient = getAllowAllHttpClient(); > + } else { > + httpClient = HttpClients.createDefault(); > + } > + > + RequestConfig requestConfig = null; > + if (UtilValidate.isNotEmpty(socketTimeout) && UtilValidate.isNotEmpty(connectionTimeout)) { > + requestConfig = RequestConfig.custom() > + .setSocketTimeout(socketTimeout) > + .setConnectTimeout(connectionTimeout) > + .setRedirectsEnabled(true) > + .build(); > + } else if (UtilValidate.isNotEmpty(socketTimeout)) { > + requestConfig = RequestConfig.custom() > + .setSocketTimeout(socketTimeout) > + .setRedirectsEnabled(true) > + .build(); > + } else if (UtilValidate.isNotEmpty(connectionTimeout)) { > + requestConfig = RequestConfig.custom() > + .setConnectTimeout(connectionTimeout) > + .setRedirectsEnabled(true) > + .build(); > + } else { > + requestConfig = RequestConfig.custom() > + .setRedirectsEnabled(true) > + .build(); > + } > + > + HttpGet httpLogin = new HttpGet(solrUrl + "/control/login?USERNAME=" + clientUsername + "&PASSWORD=" + clientPassword); > + httpLogin.setConfig(requestConfig); > + CloseableHttpResponse loginResponse = httpClient.execute(httpLogin, httpContext); > + loginResponse.close(); > + return new HttpSolrClient(solrUrl + "/" + solrIndexName, httpClient); > + } > + > } > > > > |
Thank you Jacques!
I'll check the problem you mentioned. -----邮件原件----- 发件人: Jacques Le Roux [mailto:[hidden email]] 发送时间: 2016年2月14日 22:12 收件人: [hidden email] 主题: Re: svn commit: r1730320 - in /ofbiz/trunk/specialpurpose/solr: config/ home/solrdefault/conf/ lib/runtime/ servicedef/ src/org/ofbiz/solr/ webapp/solr/WEB-INF/lib/ Sorry Jinghai With this commit a lot of tests don't pass Must be related with the httpmime-4.4.1.jar change or a dependency on solr component, I did not check Jacques Le 14/02/2016 12:59, [hidden email] a écrit : > Author: shijh > Date: Sun Feb 14 11:59:59 2016 > New Revision: 1730320 > > URL: http://svn.apache.org/viewvc?rev=1730320&view=rev > Log: > Refs OFBIZ-6715 Solr rebuild problem > > Added: > ofbiz/trunk/specialpurpose/solr/lib/runtime/httpmime-4.4.1.jar (with props) > Removed: > ofbiz/trunk/specialpurpose/solr/webapp/solr/WEB-INF/lib/httpmime-4.4.1.jar > Modified: > ofbiz/trunk/specialpurpose/solr/config/solrconfig.properties > ofbiz/trunk/specialpurpose/solr/home/solrdefault/conf/schema.xml > ofbiz/trunk/specialpurpose/solr/servicedef/solrservices.xml > ofbiz/trunk/specialpurpose/solr/src/org/ofbiz/solr/SolrProductSearch.java > ofbiz/trunk/specialpurpose/solr/src/org/ofbiz/solr/SolrUtil.java > > Modified: ofbiz/trunk/specialpurpose/solr/config/solrconfig.properties > URL: http://svn.apache.org/viewvc/ofbiz/trunk/specialpurpose/solr/config/solrconfig.properties?rev=1730320&r1=1730319&r2=1730320&view=diff > ============================================================================== > --- ofbiz/trunk/specialpurpose/solr/config/solrconfig.properties (original) > +++ ofbiz/trunk/specialpurpose/solr/config/solrconfig.properties Sun Feb 14 11:59:59 2016 > @@ -19,7 +19,7 @@ > # Solr configuration for custom OFBiz solr modules > > # Webapp access details > -solr.webapp.protocol=http > +solr.webapp.protocol=https > solr.webapp.domainName=localhost > # By default, port is same as OFBiz server, but can be overridden here. > solr.webapp.portOverride= > @@ -33,4 +33,15 @@ solr.eca.enabled=false > > # If true, connection errors during ECAs/SECAs are treated as warnings/failures rather than errors. > # If false, connection errors are treated as errors, and parent transactions are aborted. > -solr.eca.treatConnectErrorNonFatal=true > \ No newline at end of file > +solr.eca.treatConnectErrorNonFatal=true > + > +# The username and password of a solr client to CRUD an index > +solr.client.username=admin > +solr.client.password=ofbiz > + > +# Socket and connection timeout of a solr client > +solr.client.socket.timeout= > +solr.client.connection.timeout= > + > +# If true, trust self signed certification, default is false. > +solr.client.trust.selfsigned.cert=true > > Modified: ofbiz/trunk/specialpurpose/solr/home/solrdefault/conf/schema.xml > URL: http://svn.apache.org/viewvc/ofbiz/trunk/specialpurpose/solr/home/solrdefault/conf/schema.xml?rev=1730320&r1=1730319&r2=1730320&view=diff > ============================================================================== > --- ofbiz/trunk/specialpurpose/solr/home/solrdefault/conf/schema.xml (original) > +++ ofbiz/trunk/specialpurpose/solr/home/solrdefault/conf/schema.xml Sun Feb 14 11:59:59 2016 > @@ -45,7 +45,7 @@ > that avoids logging every request > --> > > -<schema name="example" version="1.5"> > +<schema name="products" version="1.5"> > <!-- attribute "name" is the name of this schema and is only used for display purposes. > version="x.y" is Solr's version number for the schema syntax and > semantics. It should not normally be changed by applications. > @@ -127,20 +127,27 @@ > in a compatible way. Any analysis applied to the <uniqueKey> should _not_ produce multiple > tokens > --> > - <field name="id" type="string" indexed="true" stored="true" required="true" multiValued="false" /> > + <!-- <field name="id" type="string" indexed="true" stored="true" required="true" multiValued="false" /> --> > > - <field name="sku" type="text_en_splitting_tight" indexed="true" stored="true" omitNorms="true"/> > - <field name="name" type="text_general" indexed="true" stored="true"/> > + <field name="productId" type="string" indexed="true" stored="true" required="true" multiValued="false" /> > + > + <field name="sku" type="text_general" indexed="true" stored="true"/> > + <field name="internalName" type="text_general" indexed="true" stored="true" multiValued="true"/> > + <field name="alphaNameSort" type="alphaOnlySort" indexed="true" stored="false"/> > <field name="manu" type="text_general" indexed="true" stored="true" omitNorms="true"/> > <field name="cat" type="string" indexed="true" stored="true" multiValued="true"/> > + <field name="catalog" type="string" indexed="true" stored="true" multiValued="true"/> > <field name="features" type="text_general" indexed="true" stored="true" multiValued="true"/> > - <field name="includes" type="text_general" indexed="true" stored="true" termVectors="true" termPositions="true" termOffsets="true" /> > - > + <field name="attributes" type="text_general" indexed="true" stored="true" multiValued="true"/> > <field name="weight" type="float" indexed="true" stored="true"/> > - <field name="price" type="float" indexed="true" stored="true"/> > + <field name="listPrice" type="float" indexed="true" stored="true"/> > + <field name="defaultPrice" type="float" indexed="true" stored="true"/> > <field name="popularity" type="int" indexed="true" stored="true" /> > - <field name="inStock" type="boolean" indexed="true" stored="true" /> > - > + <field name="inStock" type="int" indexed="true" stored="true" /> > + <field name="isVirtual" type="boolean" indexed="true" stored="true" /> > + <field name="isDigital" type="boolean" indexed="true" stored="true" /> > + <field name="isPhysical" type="boolean" indexed="true" stored="true" /> > + > <field name="store" type="location" indexed="true" stored="true"/> > > <!-- Common metadata fields, named specifically to match up with > @@ -152,18 +159,27 @@ > "resourcename": From SolrCell request param resource.name > --> > <field name="title" type="text_general" indexed="true" stored="true" multiValued="true"/> > + <field name="title_i18n_en" type="text_general" indexed="true" stored="true" multiValued="true"/> > + <field name="title_i18n_de" type="text_general" indexed="true" stored="true" multiValued="true"/> > + <field name="title_i18n_fr" type="text_general" indexed="true" stored="true" multiValued="true"/> > <field name="subject" type="text_general" indexed="true" stored="true"/> > <field name="description" type="text_general" indexed="true" stored="true"/> > + <field name="description_i18n_en" type="text_general" indexed="true" stored="true"/> > + <field name="description_i18n_de" type="text_general" indexed="true" stored="true"/> > + <field name="description_i18n_fr" type="text_general" indexed="true" stored="true"/> > + <field name="longdescription" type="text_general" indexed="true" stored="true"/> > + <field name="longdescription_i18n_en" type="text_general" indexed="true" stored="true"/> > + <field name="longdescription_i18n_de" type="text_general" indexed="true" stored="true"/> > + <field name="longdescription_i18n_fr" type="text_general" indexed="true" stored="true"/> > <field name="comments" type="text_general" indexed="true" stored="true"/> > <field name="author" type="text_general" indexed="true" stored="true"/> > <field name="keywords" type="text_general" indexed="true" stored="true"/> > - <field name="category" type="text_general" indexed="true" stored="true"/> > - <field name="resourcename" type="text_general" indexed="true" stored="true"/> > - <field name="url" type="text_general" indexed="true" stored="true"/> > <field name="content_type" type="string" indexed="true" stored="true" multiValued="true"/> > <field name="last_modified" type="date" indexed="true" stored="true"/> > <field name="links" type="string" indexed="true" stored="true" multiValued="true"/> > - <field name="_src_" type="string" indexed="false" stored="true"/> > + <field name="smallImage" type="text_general" indexed="true" stored="true"/> > + <field name="mediumImage" type="text_general" indexed="true" stored="true"/> > + <field name="largeImage" type="text_general" indexed="true" stored="true"/> > > <!-- Main body of document extracted by SolrCell. > NOTE: This field is not indexed by default, since it is also copied to "text" > @@ -255,7 +271,7 @@ > <!-- Field to use to determine and enforce document uniqueness. > Unless this field is marked with required="false", it will be a required field > --> > - <uniqueKey>id</uniqueKey> > + <uniqueKey>productId</uniqueKey> > > <!-- DEPRECATED: The defaultSearchField is consulted by various query parsers when > parsing a query string that isn't explicit about the field. Machine (non-user) > @@ -277,28 +293,15 @@ > or to add multiple fields to the same field for easier/faster searching. --> > > <copyField source="cat" dest="text"/> > - <copyField source="name" dest="text"/> > + <copyField source="internalName" dest="text"/> > <copyField source="manu" dest="text"/> > <copyField source="features" dest="text"/> > - <copyField source="includes" dest="text"/> > + <copyField source="attributes" dest="text"/> > + <copyField source="*_i18n_en" dest="text" /> > + <copyField source="*_i18n_de" dest="text" /> > + <copyField source="*_i18n_fr" dest="text" /> > <copyField source="manu" dest="manu_exact"/> > > - <!-- Copy the price into a currency enabled field (default USD) --> > - <copyField source="price" dest="price_c"/> > - > - <!-- Text fields from SolrCell to search by default in our catch-all field --> > - <copyField source="title" dest="text"/> > - <copyField source="author" dest="text"/> > - <copyField source="description" dest="text"/> > - <copyField source="keywords" dest="text"/> > - <copyField source="content" dest="text"/> > - <copyField source="content_type" dest="text"/> > - <copyField source="resourcename" dest="text"/> > - <copyField source="url" dest="text"/> > - > - <!-- Create a string version of author for faceting --> > - <copyField source="author" dest="author_s"/> > - > <!-- Above, multiple source fields are copied to the [text] field. > Another way to map multiple source fields to the same > destination field is to use the dynamic field syntax. > > Added: ofbiz/trunk/specialpurpose/solr/lib/runtime/httpmime-4.4.1.jar > URL: http://svn.apache.org/viewvc/ofbiz/trunk/specialpurpose/solr/lib/runtime/httpmime-4.4.1.jar?rev=1730320&view=auto > ============================================================================== > Binary file - no diff available. > > Propchange: ofbiz/trunk/specialpurpose/solr/lib/runtime/httpmime-4.4.1.jar > ------------------------------------------------------------------------------ > svn:mime-type = application/octet-stream > > Modified: ofbiz/trunk/specialpurpose/solr/servicedef/solrservices.xml > URL: http://svn.apache.org/viewvc/ofbiz/trunk/specialpurpose/solr/servicedef/solrservices.xml?rev=1730320&r1=1730319&r2=1730320&view=diff > ============================================================================== > --- ofbiz/trunk/specialpurpose/solr/servicedef/solrservices.xml (original) > +++ ofbiz/trunk/specialpurpose/solr/servicedef/solrservices.xml Sun Feb 14 11:59:59 2016 > @@ -24,20 +24,13 @@ under the License. > <description>Content Component Services</description> > <vendor>OFBiz</vendor> > > - <!-- Test Case --> > - <service name="SolrProductSearch" engine="java" > - transaction-timeout="72000" > - location="org.ofbiz.solr.SolrProductSearch" > - invoke="SolrProductSearch" debug="true" validate="true"> > - <description>First Test with Solr</description> > - </service> > - > <!-- Rebuild the Solr Tree --> > <service name="rebuildSolrIndex" engine="java" > transaction-timeout="72000" > location="org.ofbiz.solr.SolrProductSearch" > invoke="rebuildSolrIndex" debug="true" validate="true"> > <description>rebuild SOLR Index</description> > + <attribute mode="IN" name="indexName" optional="false" type="String"/> > <attribute mode="IN" name="treatConnectErrorNonFatal" optional="true" type="Boolean" default-value="false" /> > </service> > > @@ -48,6 +41,7 @@ under the License. > invoke="addToSolr" debug="true" validate="true"> > <description>Adds product to solr, with product denoted by productId field in instance attribute > - intended for use with ECAs/SECAs</description> > + <attribute mode="IN" name="indexName" optional="false" type="String"/> > <attribute mode="IN" name="instance" optional="false" type="org.ofbiz.entity.GenericValue" /> > </service> > > @@ -56,6 +50,7 @@ under the License. > location="org.ofbiz.solr.SolrProductSearch" > invoke="addToSolrIndex" debug="true" validate="true"> > <description>Add a Product to Solr Index</description> > + <attribute mode="IN" name="indexName" optional="false" type="String"/> > <attribute mode="IN" name="treatConnectErrorNonFatal" optional="true" type="Boolean" /> > <attribute mode="IN" name="productId" optional="false" type="String" /> > <attribute mode="IN" name="sku" optional="true" type="String" /> > @@ -88,6 +83,7 @@ under the License. > location="org.ofbiz.solr.SolrProductSearch" > invoke="addListToSolrIndex" debug="true" validate="true"> > <description>Add a List of Products to Solr Index and flush after all have been added</description> > + <attribute mode="IN" name="indexName" optional="false" type="String"/> > <attribute mode="IN" name="treatConnectErrorNonFatal" optional="true" type="Boolean" /> > <attribute mode="IN" name="fieldList" optional="false" type="List" /> > <attribute mode="OUT" name="errorType" optional="true" type="String" /> > @@ -99,6 +95,7 @@ under the License. > location="org.ofbiz.solr.SolrProductSearch" > invoke="productsSearch" debug="true" validate="true"> > <description>Run a query on Solr and return the results</description> > + <attribute mode="IN" name="indexName" optional="false" type="String"/> > <attribute mode="IN" optional="false" name="productCategoryId" type="String"/> > <attribute mode="IN" optional="true" name="viewSize" type="String"/> > <attribute mode="IN" optional="true" name="viewIndex" type="String"/> > @@ -115,6 +112,7 @@ under the License. > location="org.ofbiz.solr.SolrProductSearch" > invoke="keywordSearch" debug="true" validate="true"> > <description>Run a query on Solr and return the results</description> > + <attribute mode="IN" name="indexName" optional="false" type="String"/> > <attribute mode="IN" optional="false" name="query" type="String"/> > <attribute mode="IN" optional="true" name="viewSize" type="String"/> > <attribute mode="IN" optional="true" name="viewIndex" type="String"/> > @@ -139,6 +137,7 @@ under the License. > location="org.ofbiz.solr.SolrProductSearch" > invoke="runSolrQuery" debug="true" validate="true"> > <description>Run a query on Solr and return the results</description> > + <attribute mode="IN" name="indexName" optional="false" type="String"/> > <attribute mode="IN" optional="false" name="query" type="String"/> > <attribute mode="IN" optional="true" name="viewSize" type="Integer"/> > <attribute mode="IN" optional="true" name="viewIndex" type="Integer"/> > @@ -158,6 +157,7 @@ under the License. > location="org.ofbiz.solr.SolrProductSearch" > invoke="getAvailableCategories" debug="true" validate="true"> > <description>Run a query on Solr and return the results</description> > + <attribute mode="IN" name="indexName" optional="false" type="String"/> > <attribute mode="IN" optional="true" name="productCategoryId" type="String"/> > <attribute mode="IN" optional="true" name="productId" type="String"/> > <attribute mode="IN" optional="true" name="catalogId" type="String"/> > @@ -173,6 +173,7 @@ under the License. > location="org.ofbiz.solr.SolrProductSearch" > invoke="getSideDeepCategories" debug="true" validate="true"> > <description>Run a query on Solr and return the results</description> > + <attribute mode="IN" name="indexName" optional="false" type="String"/> > <attribute mode="IN" optional="true" name="productCategoryId" type="String"/> > <attribute mode="IN" optional="true" name="catalogId" type="String"/> > <attribute name="numFound" type="Long" mode="OUT" optional="false"/> > > Modified: ofbiz/trunk/specialpurpose/solr/src/org/ofbiz/solr/SolrProductSearch.java > URL: http://svn.apache.org/viewvc/ofbiz/trunk/specialpurpose/solr/src/org/ofbiz/solr/SolrProductSearch.java?rev=1730320&r1=1730319&r2=1730320&view=diff > ============================================================================== > --- ofbiz/trunk/specialpurpose/solr/src/org/ofbiz/solr/SolrProductSearch.java (original) > +++ ofbiz/trunk/specialpurpose/solr/src/org/ofbiz/solr/SolrProductSearch.java Sun Feb 14 11:59:59 2016 > @@ -62,7 +62,6 @@ public abstract class SolrProductSearch > > public static final String module = SolrProductSearch.class.getName(); > > - > /** > * Adds product to solr, with product denoted by productId field in instance attribute > * - intended for use with ECAs/SECAs. > @@ -73,6 +72,7 @@ public abstract class SolrProductSearch > Delegator delegator = dctx.getDelegator(); > GenericValue productInstance = (GenericValue) context.get("instance"); > String productId = (String) productInstance.get("productId"); > + String solrIndexName = (String) context.get("indexName"); > > if (SolrUtil.isSolrEcaEnabled()) { > // Debug.logVerbose("Solr: addToSolr: Running indexing for productId '" + productId + "'", module); > @@ -80,6 +80,7 @@ public abstract class SolrProductSearch > GenericValue product = delegator.findOne("Product", UtilMisc.toMap("productId", productId), false); > Map<String, Object> dispatchContext = ProductUtil.getProductContent(product, dctx, context); > dispatchContext.put("treatConnectErrorNonFatal", SolrUtil.isEcaTreatConnectErrorNonFatal()); > + dispatchContext.put("indexName", solrIndexName); > Map<String, Object> runResult = dispatcher.runSync("addToSolrIndex", dispatchContext); > String runMsg = ServiceUtil.getErrorMessage(runResult); > if (UtilValidate.isEmpty(runMsg)) { > @@ -114,13 +115,14 @@ public abstract class SolrProductSearch > HttpSolrClient client = null; > Map<String, Object> result; > String productId = (String) context.get("productId"); > + String solrIndexName = (String) context.get("indexName"); > // connectErrorNonFatal is a necessary option because in some cases it may be considered normal that solr server is unavailable; > // don't want to return error and abort transactions in these cases. > Boolean treatConnectErrorNonFatal = (Boolean) context.get("treatConnectErrorNonFatal"); > try { > Debug.logInfo("Solr: Generating and indexing document for productId '" + productId + "'", module); > > - client = new HttpSolrClient(SolrUtil.solrUrl); > + client = SolrUtil.getInstance().getHttpSolrClient(solrIndexName); > //Debug.log(server.ping().toString()); > > // Construct Documents > @@ -185,6 +187,7 @@ public abstract class SolrProductSearch > * This is faster than reflushing the index each time. > */ > public static Map<String, Object> addListToSolrIndex(DispatchContext dctx, Map<String, Object> context) throws GenericEntityException { > + String solrIndexName = (String) context.get("indexName"); > HttpSolrClient client = null; > Map<String, Object> result; > Boolean treatConnectErrorNonFatal = (Boolean) context.get("treatConnectErrorNonFatal"); > @@ -204,7 +207,7 @@ public abstract class SolrProductSearch > docs.add(doc1); > } > // push Documents to server > - client = new HttpSolrClient(SolrUtil.solrUrl); > + client = SolrUtil.getInstance().getHttpSolrClient(solrIndexName); > client.add(docs); > client.commit(); > > @@ -258,9 +261,10 @@ public abstract class SolrProductSearch > public static Map<String, Object> runSolrQuery(DispatchContext dctx, Map<String, Object> context) { > // get Connection > HttpSolrClient client = null; > + String solrIndexName = (String) context.get("indexName"); > Map<String, Object> result; > try { > - client = new HttpSolrClient(SolrUtil.solrUrl); > + client = SolrUtil.getInstance().getHttpSolrClient(solrIndexName); > // create Query Object > SolrQuery solrQuery = new SolrQuery(); > solrQuery.setQuery((String) context.get("query")); > @@ -363,6 +367,7 @@ public abstract class SolrProductSearch > public static Map<String, Object> productsSearch(DispatchContext dctx, Map<String, Object> context) { > Map<String, Object> result; > LocalDispatcher dispatcher = dctx.getDispatcher(); > + String solrIndexName = (String) context.get("indexName"); > > try { > Map<String, Object> dispatchMap = new HashMap<String, Object>(); > @@ -381,6 +386,7 @@ public abstract class SolrProductSearch > dispatchMap.put("facet", false); > dispatchMap.put("spellcheck", true); > dispatchMap.put("highlight", true); > + dispatchMap.put("indexName", solrIndexName); > Map<String, Object> searchResult = dispatcher.runSync("runSolrQuery", dispatchMap); > QueryResponse queryResult = (QueryResponse) searchResult.get("queryResult"); > result = ServiceUtil.returnSuccess(); > @@ -403,6 +409,7 @@ public abstract class SolrProductSearch > public static Map<String, Object> keywordSearch(DispatchContext dctx, Map<String, Object> context) { > Map<String, Object> result; > LocalDispatcher dispatcher = dctx.getDispatcher(); > + String solrIndexName = (String) context.get("indexName"); > > try { > if (context.get("query") == null || context.get("query").equals("")) > @@ -418,6 +425,8 @@ public abstract class SolrProductSearch > if (context.get("queryFilter") != null) > dispatchMap.put("queryFilter", context.get("queryFilter")); > dispatchMap.put("spellcheck", true); > + dispatchMap.put("indexName", solrIndexName); > + > Map<String, Object> searchResult = dispatcher.runSync("runSolrQuery", dispatchMap); > QueryResponse queryResult = (QueryResponse) searchResult.get("queryResult"); > > @@ -479,6 +488,7 @@ public abstract class SolrProductSearch > */ > public static Map<String, Object> getAvailableCategories(DispatchContext dctx, Map<String, Object> context) { > Map<String, Object> result; > + String solrIndexName = (String) context.get("indexName"); > try { > boolean displayProducts = false; > if (UtilValidate.isNotEmpty(context.get("displayProducts"))) > @@ -494,10 +504,9 @@ public abstract class SolrProductSearch > if (UtilValidate.isNotEmpty(context.get("catalogId"))) > catalogId = (String) context.get("catalogId"); > > - //String productCategoryId = (String) context.get("productCategoryId") != null ? CategoryUtil.getCategoryNameWithTrail((String) context.get("productCategoryId"), dctx): null; > String productCategoryId = (String) context.get("productCategoryId") != null ? CategoryUtil.getCategoryNameWithTrail((String) context.get("productCategoryId"),dctx) : null; > - Debug.logInfo("productCategoryId "+productCategoryId, module); > - Map<String, Object> query = SolrUtil.categoriesAvailable(catalogId, productCategoryId, (String) context.get("productId"), displayProducts, viewIndex, viewSize); > + Debug.logInfo("productCategoryId " + productCategoryId, module); > + Map<String, Object> query = SolrUtil.categoriesAvailable(catalogId, productCategoryId, (String) context.get("productId"), displayProducts, viewIndex, viewSize, solrIndexName); > > QueryResponse cat = (QueryResponse) query.get("rows"); > result = ServiceUtil.returnSuccess(); > @@ -508,7 +517,7 @@ public abstract class SolrProductSearch > FacetField field = (FacetField) catIterator.next(); > List<Count> catL = (List<Count>) field.getValues(); > if (catL != null) { > - // log.info("FacetFields = "+catL); > + // Debug.logInfo("FacetFields = " + catL, module); > for (Iterator<Count> catIter = catL.iterator(); catIter.hasNext();) { > FacetField.Count f = (FacetField.Count) catIter.next(); > if (f.getCount() > 0) { > @@ -517,7 +526,7 @@ public abstract class SolrProductSearch > } > result.put("categories", categories); > result.put("numFound", cat.getResults().getNumFound()); > - // log.info("The returned map is this:"+result); > + // Debug.logInfo("The returned map is this:" + result, module); > } > } > } catch (Exception e) { > @@ -533,6 +542,7 @@ public abstract class SolrProductSearch > */ > public static Map<String, Object> getSideDeepCategories(DispatchContext dctx, Map<String, Object> context) { > Map<String, Object> result; > + String solrIndexName = (String) context.get("indexName"); > try { > String catalogId = null; > if (UtilValidate.isNotEmpty(context.get("catalogId"))) > @@ -556,7 +566,7 @@ public abstract class SolrProductSearch > int level = Integer.parseInt(categoryPathArray[0]); > String facetQuery = CategoryUtil.getFacetFilterForCategory(categoryPath, dctx); > //Debug.logInfo("categoryPath: "+categoryPath + " facetQuery: "+facetQuery,module); > - Map<String, Object> query = SolrUtil.categoriesAvailable(catalogId, categoryPath, null, facetQuery,false, 0, 0); > + Map<String, Object> query = SolrUtil.categoriesAvailable(catalogId, categoryPath, null, facetQuery, false, 0, 0, solrIndexName); > QueryResponse cat = (QueryResponse) query.get("rows"); > List<Map<String, Object>> categories = new ArrayList<Map<String, Object>>(); > > @@ -612,12 +622,13 @@ public abstract class SolrProductSearch > GenericDelegator delegator = (GenericDelegator) dctx.getDelegator(); > LocalDispatcher dispatcher = dctx.getDispatcher(); > GenericValue userLogin = (GenericValue) context.get("userLogin"); > - Locale locale = new Locale("de_DE"); > + Locale locale = (Locale) context.get("locale"); > + String solrIndexName = (String) context.get("indexName"); > > Boolean treatConnectErrorNonFatal = (Boolean) context.get("treatConnectErrorNonFatal"); > > try { > - client = new HttpSolrClient(SolrUtil.solrUrl); > + client = SolrUtil.getInstance().getHttpSolrClient(solrIndexName); > > // now lets fetch all products > List<Map<String, Object>> solrDocs = new ArrayList<Map<String, Object>>(); > @@ -642,7 +653,7 @@ public abstract class SolrProductSearch > > // THis adds all products to the Index (instantly) > Map<String, Object> runResult = dispatcher.runSync("addListToSolrIndex", UtilMisc.toMap("fieldList", solrDocs, "userLogin", userLogin, > - "locale", locale, "treatConnectErrorNonFatal", treatConnectErrorNonFatal)); > + "locale", locale, "indexName", solrIndexName, "treatConnectErrorNonFatal", treatConnectErrorNonFatal)); > > String runMsg = ServiceUtil.getErrorMessage(runResult); > if (UtilValidate.isEmpty(runMsg)) { > > Modified: ofbiz/trunk/specialpurpose/solr/src/org/ofbiz/solr/SolrUtil.java > URL: http://svn.apache.org/viewvc/ofbiz/trunk/specialpurpose/solr/src/org/ofbiz/solr/SolrUtil.java?rev=1730320&r1=1730319&r2=1730320&view=diff > ============================================================================== > --- ofbiz/trunk/specialpurpose/solr/src/org/ofbiz/solr/SolrUtil.java (original) > +++ ofbiz/trunk/specialpurpose/solr/src/org/ofbiz/solr/SolrUtil.java Sun Feb 14 11:59:59 2016 > @@ -18,12 +18,26 @@ > *******************************************************************************/ > package org.ofbiz.solr; > > +import java.io.IOException; > import java.util.HashMap; > import java.util.Iterator; > import java.util.List; > import java.util.Map; > import java.util.Set; > > +import javax.net.ssl.SSLContext; > + > +import org.apache.http.client.ClientProtocolException; > +import org.apache.http.client.config.RequestConfig; > +import org.apache.http.client.methods.CloseableHttpResponse; > +import org.apache.http.client.methods.HttpGet; > +import org.apache.http.client.protocol.HttpClientContext; > +import org.apache.http.conn.ssl.NoopHostnameVerifier; > +import org.apache.http.conn.ssl.SSLConnectionSocketFactory; > +import org.apache.http.ssl.SSLContexts; > +import org.apache.http.conn.ssl.TrustSelfSignedStrategy; > +import org.apache.http.impl.client.CloseableHttpClient; > +import org.apache.http.impl.client.HttpClients; > import org.apache.solr.client.solrj.SolrQuery; > import org.apache.solr.client.solrj.SolrRequest.METHOD; > import org.apache.solr.client.solrj.impl.HttpSolrClient; > @@ -33,6 +47,7 @@ import org.ofbiz.base.component.Componen > import org.ofbiz.base.component.ComponentConfig.WebappInfo; > import org.ofbiz.base.component.ComponentException; > import org.ofbiz.base.util.Debug; > +import org.ofbiz.base.util.FileUtil; > import org.ofbiz.base.util.UtilGenerics; > import org.ofbiz.base.util.UtilProperties; > import org.ofbiz.base.util.UtilValidate; > @@ -41,7 +56,7 @@ import org.ofbiz.entity.GenericEntityExc > /** > * Solr utility class. > */ > -public abstract class SolrUtil { > +public final class SolrUtil { > > public static final String module = SolrUtil.class.getName(); > private static String[] solrProdAttribute = { "productId", "internalName", "manu", "size", "smallImage", "mediumImage", "largeImage", "listPrice", "defaultPrice", "inStock", "isVirtual" }; > @@ -49,7 +64,27 @@ public abstract class SolrUtil { > public static final String solrConfigName = "solrconfig.properties"; > public static final String solrUrl = makeSolrWebappUrl(); > > - public static String makeSolrWebappUrl() { > + protected static final String socketTimeoutString = UtilProperties.getPropertyValue(solrConfigName, "solr.client.socket.timeout"); > + > + protected static final String connectionTimeoutString = UtilProperties.getPropertyValue(solrConfigName, "solr.client.connection.timeout"); > + > + protected static final String clientUsername = UtilProperties.getPropertyValue(solrConfigName, "solr.client.username"); > + > + protected static final String clientPassword = UtilProperties.getPropertyValue(solrConfigName, "solr.client.password"); > + > + protected static final Integer socketTimeout = getSocketTimeout(); > + > + protected static final Integer connectionTimeout = getConnectionTimeout(); > + > + protected static final String trustSelfSignedCertString = UtilProperties.getPropertyValue(solrConfigName, "solr.client.trust.selfsigned.cert", "false"); > + > + protected static final boolean trustSelfSignedCert = getTrustSelfSignedCert(); > + > + protected SolrUtil() { > + // empty constructor > + } > + > + public static String makeSolrWebappUrl() { > final String solrWebappProtocol = UtilProperties.getPropertyValue(solrConfigName, "solr.webapp.protocol"); > final String solrWebappDomainName = UtilProperties.getPropertyValue(solrConfigName, "solr.webapp.domainName"); > final String solrWebappPath = UtilProperties.getPropertyValue(solrConfigName, "solr.webapp.path"); > @@ -58,14 +93,42 @@ public abstract class SolrUtil { > String solrPort; > if (UtilValidate.isNotEmpty(solrWebappPortOverride)) { > solrPort = solrWebappPortOverride; > - } > - else { > + } else { > solrPort = UtilProperties.getPropertyValue("url", ("https".equals(solrWebappProtocol) ? "port.https" : "port.http")); > } > > return solrWebappProtocol + "://" + solrWebappDomainName + ":" + solrPort + solrWebappPath; > } > > + private static Integer getSocketTimeout() { > + if (UtilValidate.isNotEmpty(socketTimeoutString)) { > + try { > + return Integer.parseInt(socketTimeoutString); > + } catch (Exception e) { > + return null; > + } > + } > + return null; > + } > + > + private static Integer getConnectionTimeout() { > + if (UtilValidate.isNotEmpty(connectionTimeoutString)) { > + try { > + return Integer.parseInt(connectionTimeoutString); > + } catch (Exception e) { > + return null; > + } > + } > + return null; > + } > + > + private static boolean getTrustSelfSignedCert() { > + if ("true".equals(trustSelfSignedCertString)) { > + return true; > + } > + return false; > + } > + > public static boolean isSolrEcaEnabled() { > Boolean ecaEnabled = null; > String sysProp = System.getProperty("ofbiz.solr.eca.enabled"); > @@ -181,18 +244,18 @@ public abstract class SolrUtil { > return doc1; > } > > - public static Map<String, Object> categoriesAvailable(String catalogId, String categoryId, String productId, boolean displayproducts, int viewIndex, int viewSize) { > - return categoriesAvailable(catalogId,categoryId,productId,null,displayproducts,viewIndex,viewSize); > + public static Map<String, Object> categoriesAvailable(String catalogId, String categoryId, String productId, boolean displayproducts, int viewIndex, int viewSize, String solrIndexName) { > + return categoriesAvailable(catalogId, categoryId, productId, null, displayproducts, viewIndex, viewSize, solrIndexName); > } > > - public static Map<String, Object> categoriesAvailable(String catalogId, String categoryId, String productId, String facetPrefix, boolean displayproducts, int viewIndex, int viewSize) { > + public static Map<String, Object> categoriesAvailable(String catalogId, String categoryId, String productId, String facetPrefix, boolean displayproducts, int viewIndex, int viewSize, String solrIndexName) { > // create the data model > Map<String, Object> result = new HashMap<String, Object>(); > HttpSolrClient client = null; > QueryResponse returnMap = new QueryResponse(); > try { > // do the basic query > - client = new HttpSolrClient(solrUrl); > + client = getInstance().getHttpSolrClient(solrIndexName); > // create Query Object > String query = "inStock[1 TO *]"; > if (categoryId != null) > @@ -235,4 +298,68 @@ public abstract class SolrUtil { > return result; > } > > + private CloseableHttpClient getAllowAllHttpClient() { > + try { > + // Trust own CA and all self-signed certs > + SSLContext sslContext = SSLContexts.custom() > + .loadTrustMaterial(FileUtil.getFile("component://base/config/ofbizssl.jks"), "changeit".toCharArray(), > + new TrustSelfSignedStrategy()) > + .build(); > + // No host name verifier > + SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory( > + sslContext, > + NoopHostnameVerifier.INSTANCE); > + CloseableHttpClient httpClient = HttpClients.custom() > + .setSSLSocketFactory(sslsf) > + .build(); > + return httpClient; > + } catch (Exception e) { > + return HttpClients.createDefault(); > + } > + } > + > + public static SolrUtil getInstance() { > + return new SolrUtil(); > + } > + > + public HttpSolrClient getHttpSolrClient(String solrIndexName) throws ClientProtocolException, IOException { > + HttpClientContext httpContext = HttpClientContext.create(); > + > + CloseableHttpClient httpClient = null; > + if (trustSelfSignedCert) { > + httpClient = getAllowAllHttpClient(); > + } else { > + httpClient = HttpClients.createDefault(); > + } > + > + RequestConfig requestConfig = null; > + if (UtilValidate.isNotEmpty(socketTimeout) && UtilValidate.isNotEmpty(connectionTimeout)) { > + requestConfig = RequestConfig.custom() > + .setSocketTimeout(socketTimeout) > + .setConnectTimeout(connectionTimeout) > + .setRedirectsEnabled(true) > + .build(); > + } else if (UtilValidate.isNotEmpty(socketTimeout)) { > + requestConfig = RequestConfig.custom() > + .setSocketTimeout(socketTimeout) > + .setRedirectsEnabled(true) > + .build(); > + } else if (UtilValidate.isNotEmpty(connectionTimeout)) { > + requestConfig = RequestConfig.custom() > + .setConnectTimeout(connectionTimeout) > + .setRedirectsEnabled(true) > + .build(); > + } else { > + requestConfig = RequestConfig.custom() > + .setRedirectsEnabled(true) > + .build(); > + } > + > + HttpGet httpLogin = new HttpGet(solrUrl + "/control/login?USERNAME=" + clientUsername + "&PASSWORD=" + clientPassword); > + httpLogin.setConfig(requestConfig); > + CloseableHttpResponse loginResponse = httpClient.execute(httpLogin, httpContext); > + loginResponse.close(); > + return new HttpSolrClient(solrUrl + "/" + solrIndexName, httpClient); > + } > + > } > > > > |
Administrator
|
Sorry Jinghai,
I could not wait, I need to have the tests clean to move ahead. So I reverted r1730320 at r1730873 Jacques Le 15/02/2016 03:51, Shi Jinghai a écrit : > Thank you Jacques! > > I'll check the problem you mentioned. > > -----邮件原件----- > 发件人: Jacques Le Roux [mailto:[hidden email]] > 发送时间: 2016年2月14日 22:12 > 收件人: [hidden email] > 主题: Re: svn commit: r1730320 - in /ofbiz/trunk/specialpurpose/solr: config/ home/solrdefault/conf/ lib/runtime/ servicedef/ src/org/ofbiz/solr/ webapp/solr/WEB-INF/lib/ > > Sorry Jinghai > > With this commit a lot of tests don't pass > Must be related with the httpmime-4.4.1.jar change or a dependency on solr component, I did not check > > Jacques > > Le 14/02/2016 12:59, [hidden email] a écrit : >> Author: shijh >> Date: Sun Feb 14 11:59:59 2016 >> New Revision: 1730320 >> >> URL: http://svn.apache.org/viewvc?rev=1730320&view=rev >> Log: >> Refs OFBIZ-6715 Solr rebuild problem >> >> Added: >> ofbiz/trunk/specialpurpose/solr/lib/runtime/httpmime-4.4.1.jar (with props) >> Removed: >> ofbiz/trunk/specialpurpose/solr/webapp/solr/WEB-INF/lib/httpmime-4.4.1.jar >> Modified: >> ofbiz/trunk/specialpurpose/solr/config/solrconfig.properties >> ofbiz/trunk/specialpurpose/solr/home/solrdefault/conf/schema.xml >> ofbiz/trunk/specialpurpose/solr/servicedef/solrservices.xml >> ofbiz/trunk/specialpurpose/solr/src/org/ofbiz/solr/SolrProductSearch.java >> ofbiz/trunk/specialpurpose/solr/src/org/ofbiz/solr/SolrUtil.java >> >> Modified: ofbiz/trunk/specialpurpose/solr/config/solrconfig.properties >> URL: http://svn.apache.org/viewvc/ofbiz/trunk/specialpurpose/solr/config/solrconfig.properties?rev=1730320&r1=1730319&r2=1730320&view=diff >> ============================================================================== >> --- ofbiz/trunk/specialpurpose/solr/config/solrconfig.properties (original) >> +++ ofbiz/trunk/specialpurpose/solr/config/solrconfig.properties Sun Feb 14 11:59:59 2016 >> @@ -19,7 +19,7 @@ >> # Solr configuration for custom OFBiz solr modules >> >> # Webapp access details >> -solr.webapp.protocol=http >> +solr.webapp.protocol=https >> solr.webapp.domainName=localhost >> # By default, port is same as OFBiz server, but can be overridden here. >> solr.webapp.portOverride= >> @@ -33,4 +33,15 @@ solr.eca.enabled=false >> >> # If true, connection errors during ECAs/SECAs are treated as warnings/failures rather than errors. >> # If false, connection errors are treated as errors, and parent transactions are aborted. >> -solr.eca.treatConnectErrorNonFatal=true >> \ No newline at end of file >> +solr.eca.treatConnectErrorNonFatal=true >> + >> +# The username and password of a solr client to CRUD an index >> +solr.client.username=admin >> +solr.client.password=ofbiz >> + >> +# Socket and connection timeout of a solr client >> +solr.client.socket.timeout= >> +solr.client.connection.timeout= >> + >> +# If true, trust self signed certification, default is false. >> +solr.client.trust.selfsigned.cert=true >> >> Modified: ofbiz/trunk/specialpurpose/solr/home/solrdefault/conf/schema.xml >> URL: http://svn.apache.org/viewvc/ofbiz/trunk/specialpurpose/solr/home/solrdefault/conf/schema.xml?rev=1730320&r1=1730319&r2=1730320&view=diff >> ============================================================================== >> --- ofbiz/trunk/specialpurpose/solr/home/solrdefault/conf/schema.xml (original) >> +++ ofbiz/trunk/specialpurpose/solr/home/solrdefault/conf/schema.xml Sun Feb 14 11:59:59 2016 >> @@ -45,7 +45,7 @@ >> that avoids logging every request >> --> >> >> -<schema name="example" version="1.5"> >> +<schema name="products" version="1.5"> >> <!-- attribute "name" is the name of this schema and is only used for display purposes. >> version="x.y" is Solr's version number for the schema syntax and >> semantics. It should not normally be changed by applications. >> @@ -127,20 +127,27 @@ >> in a compatible way. Any analysis applied to the <uniqueKey> should _not_ produce multiple >> tokens >> --> >> - <field name="id" type="string" indexed="true" stored="true" required="true" multiValued="false" /> >> + <!-- <field name="id" type="string" indexed="true" stored="true" required="true" multiValued="false" /> --> >> >> - <field name="sku" type="text_en_splitting_tight" indexed="true" stored="true" omitNorms="true"/> >> - <field name="name" type="text_general" indexed="true" stored="true"/> >> + <field name="productId" type="string" indexed="true" stored="true" required="true" multiValued="false" /> >> + >> + <field name="sku" type="text_general" indexed="true" stored="true"/> >> + <field name="internalName" type="text_general" indexed="true" stored="true" multiValued="true"/> >> + <field name="alphaNameSort" type="alphaOnlySort" indexed="true" stored="false"/> >> <field name="manu" type="text_general" indexed="true" stored="true" omitNorms="true"/> >> <field name="cat" type="string" indexed="true" stored="true" multiValued="true"/> >> + <field name="catalog" type="string" indexed="true" stored="true" multiValued="true"/> >> <field name="features" type="text_general" indexed="true" stored="true" multiValued="true"/> >> - <field name="includes" type="text_general" indexed="true" stored="true" termVectors="true" termPositions="true" termOffsets="true" /> >> - >> + <field name="attributes" type="text_general" indexed="true" stored="true" multiValued="true"/> >> <field name="weight" type="float" indexed="true" stored="true"/> >> - <field name="price" type="float" indexed="true" stored="true"/> >> + <field name="listPrice" type="float" indexed="true" stored="true"/> >> + <field name="defaultPrice" type="float" indexed="true" stored="true"/> >> <field name="popularity" type="int" indexed="true" stored="true" /> >> - <field name="inStock" type="boolean" indexed="true" stored="true" /> >> - >> + <field name="inStock" type="int" indexed="true" stored="true" /> >> + <field name="isVirtual" type="boolean" indexed="true" stored="true" /> >> + <field name="isDigital" type="boolean" indexed="true" stored="true" /> >> + <field name="isPhysical" type="boolean" indexed="true" stored="true" /> >> + >> <field name="store" type="location" indexed="true" stored="true"/> >> >> <!-- Common metadata fields, named specifically to match up with >> @@ -152,18 +159,27 @@ >> "resourcename": From SolrCell request param resource.name >> --> >> <field name="title" type="text_general" indexed="true" stored="true" multiValued="true"/> >> + <field name="title_i18n_en" type="text_general" indexed="true" stored="true" multiValued="true"/> >> + <field name="title_i18n_de" type="text_general" indexed="true" stored="true" multiValued="true"/> >> + <field name="title_i18n_fr" type="text_general" indexed="true" stored="true" multiValued="true"/> >> <field name="subject" type="text_general" indexed="true" stored="true"/> >> <field name="description" type="text_general" indexed="true" stored="true"/> >> + <field name="description_i18n_en" type="text_general" indexed="true" stored="true"/> >> + <field name="description_i18n_de" type="text_general" indexed="true" stored="true"/> >> + <field name="description_i18n_fr" type="text_general" indexed="true" stored="true"/> >> + <field name="longdescription" type="text_general" indexed="true" stored="true"/> >> + <field name="longdescription_i18n_en" type="text_general" indexed="true" stored="true"/> >> + <field name="longdescription_i18n_de" type="text_general" indexed="true" stored="true"/> >> + <field name="longdescription_i18n_fr" type="text_general" indexed="true" stored="true"/> >> <field name="comments" type="text_general" indexed="true" stored="true"/> >> <field name="author" type="text_general" indexed="true" stored="true"/> >> <field name="keywords" type="text_general" indexed="true" stored="true"/> >> - <field name="category" type="text_general" indexed="true" stored="true"/> >> - <field name="resourcename" type="text_general" indexed="true" stored="true"/> >> - <field name="url" type="text_general" indexed="true" stored="true"/> >> <field name="content_type" type="string" indexed="true" stored="true" multiValued="true"/> >> <field name="last_modified" type="date" indexed="true" stored="true"/> >> <field name="links" type="string" indexed="true" stored="true" multiValued="true"/> >> - <field name="_src_" type="string" indexed="false" stored="true"/> >> + <field name="smallImage" type="text_general" indexed="true" stored="true"/> >> + <field name="mediumImage" type="text_general" indexed="true" stored="true"/> >> + <field name="largeImage" type="text_general" indexed="true" stored="true"/> >> >> <!-- Main body of document extracted by SolrCell. >> NOTE: This field is not indexed by default, since it is also copied to "text" >> @@ -255,7 +271,7 @@ >> <!-- Field to use to determine and enforce document uniqueness. >> Unless this field is marked with required="false", it will be a required field >> --> >> - <uniqueKey>id</uniqueKey> >> + <uniqueKey>productId</uniqueKey> >> >> <!-- DEPRECATED: The defaultSearchField is consulted by various query parsers when >> parsing a query string that isn't explicit about the field. Machine (non-user) >> @@ -277,28 +293,15 @@ >> or to add multiple fields to the same field for easier/faster searching. --> >> >> <copyField source="cat" dest="text"/> >> - <copyField source="name" dest="text"/> >> + <copyField source="internalName" dest="text"/> >> <copyField source="manu" dest="text"/> >> <copyField source="features" dest="text"/> >> - <copyField source="includes" dest="text"/> >> + <copyField source="attributes" dest="text"/> >> + <copyField source="*_i18n_en" dest="text" /> >> + <copyField source="*_i18n_de" dest="text" /> >> + <copyField source="*_i18n_fr" dest="text" /> >> <copyField source="manu" dest="manu_exact"/> >> >> - <!-- Copy the price into a currency enabled field (default USD) --> >> - <copyField source="price" dest="price_c"/> >> - >> - <!-- Text fields from SolrCell to search by default in our catch-all field --> >> - <copyField source="title" dest="text"/> >> - <copyField source="author" dest="text"/> >> - <copyField source="description" dest="text"/> >> - <copyField source="keywords" dest="text"/> >> - <copyField source="content" dest="text"/> >> - <copyField source="content_type" dest="text"/> >> - <copyField source="resourcename" dest="text"/> >> - <copyField source="url" dest="text"/> >> - >> - <!-- Create a string version of author for faceting --> >> - <copyField source="author" dest="author_s"/> >> - >> <!-- Above, multiple source fields are copied to the [text] field. >> Another way to map multiple source fields to the same >> destination field is to use the dynamic field syntax. >> >> Added: ofbiz/trunk/specialpurpose/solr/lib/runtime/httpmime-4.4.1.jar >> URL: http://svn.apache.org/viewvc/ofbiz/trunk/specialpurpose/solr/lib/runtime/httpmime-4.4.1.jar?rev=1730320&view=auto >> ============================================================================== >> Binary file - no diff available. >> >> Propchange: ofbiz/trunk/specialpurpose/solr/lib/runtime/httpmime-4.4.1.jar >> ------------------------------------------------------------------------------ >> svn:mime-type = application/octet-stream >> >> Modified: ofbiz/trunk/specialpurpose/solr/servicedef/solrservices.xml >> URL: http://svn.apache.org/viewvc/ofbiz/trunk/specialpurpose/solr/servicedef/solrservices.xml?rev=1730320&r1=1730319&r2=1730320&view=diff >> ============================================================================== >> --- ofbiz/trunk/specialpurpose/solr/servicedef/solrservices.xml (original) >> +++ ofbiz/trunk/specialpurpose/solr/servicedef/solrservices.xml Sun Feb 14 11:59:59 2016 >> @@ -24,20 +24,13 @@ under the License. >> <description>Content Component Services</description> >> <vendor>OFBiz</vendor> >> >> - <!-- Test Case --> >> - <service name="SolrProductSearch" engine="java" >> - transaction-timeout="72000" >> - location="org.ofbiz.solr.SolrProductSearch" >> - invoke="SolrProductSearch" debug="true" validate="true"> >> - <description>First Test with Solr</description> >> - </service> >> - >> <!-- Rebuild the Solr Tree --> >> <service name="rebuildSolrIndex" engine="java" >> transaction-timeout="72000" >> location="org.ofbiz.solr.SolrProductSearch" >> invoke="rebuildSolrIndex" debug="true" validate="true"> >> <description>rebuild SOLR Index</description> >> + <attribute mode="IN" name="indexName" optional="false" type="String"/> >> <attribute mode="IN" name="treatConnectErrorNonFatal" optional="true" type="Boolean" default-value="false" /> >> </service> >> >> @@ -48,6 +41,7 @@ under the License. >> invoke="addToSolr" debug="true" validate="true"> >> <description>Adds product to solr, with product denoted by productId field in instance attribute >> - intended for use with ECAs/SECAs</description> >> + <attribute mode="IN" name="indexName" optional="false" type="String"/> >> <attribute mode="IN" name="instance" optional="false" type="org.ofbiz.entity.GenericValue" /> >> </service> >> >> @@ -56,6 +50,7 @@ under the License. >> location="org.ofbiz.solr.SolrProductSearch" >> invoke="addToSolrIndex" debug="true" validate="true"> >> <description>Add a Product to Solr Index</description> >> + <attribute mode="IN" name="indexName" optional="false" type="String"/> >> <attribute mode="IN" name="treatConnectErrorNonFatal" optional="true" type="Boolean" /> >> <attribute mode="IN" name="productId" optional="false" type="String" /> >> <attribute mode="IN" name="sku" optional="true" type="String" /> >> @@ -88,6 +83,7 @@ under the License. >> location="org.ofbiz.solr.SolrProductSearch" >> invoke="addListToSolrIndex" debug="true" validate="true"> >> <description>Add a List of Products to Solr Index and flush after all have been added</description> >> + <attribute mode="IN" name="indexName" optional="false" type="String"/> >> <attribute mode="IN" name="treatConnectErrorNonFatal" optional="true" type="Boolean" /> >> <attribute mode="IN" name="fieldList" optional="false" type="List" /> >> <attribute mode="OUT" name="errorType" optional="true" type="String" /> >> @@ -99,6 +95,7 @@ under the License. >> location="org.ofbiz.solr.SolrProductSearch" >> invoke="productsSearch" debug="true" validate="true"> >> <description>Run a query on Solr and return the results</description> >> + <attribute mode="IN" name="indexName" optional="false" type="String"/> >> <attribute mode="IN" optional="false" name="productCategoryId" type="String"/> >> <attribute mode="IN" optional="true" name="viewSize" type="String"/> >> <attribute mode="IN" optional="true" name="viewIndex" type="String"/> >> @@ -115,6 +112,7 @@ under the License. >> location="org.ofbiz.solr.SolrProductSearch" >> invoke="keywordSearch" debug="true" validate="true"> >> <description>Run a query on Solr and return the results</description> >> + <attribute mode="IN" name="indexName" optional="false" type="String"/> >> <attribute mode="IN" optional="false" name="query" type="String"/> >> <attribute mode="IN" optional="true" name="viewSize" type="String"/> >> <attribute mode="IN" optional="true" name="viewIndex" type="String"/> >> @@ -139,6 +137,7 @@ under the License. >> location="org.ofbiz.solr.SolrProductSearch" >> invoke="runSolrQuery" debug="true" validate="true"> >> <description>Run a query on Solr and return the results</description> >> + <attribute mode="IN" name="indexName" optional="false" type="String"/> >> <attribute mode="IN" optional="false" name="query" type="String"/> >> <attribute mode="IN" optional="true" name="viewSize" type="Integer"/> >> <attribute mode="IN" optional="true" name="viewIndex" type="Integer"/> >> @@ -158,6 +157,7 @@ under the License. >> location="org.ofbiz.solr.SolrProductSearch" >> invoke="getAvailableCategories" debug="true" validate="true"> >> <description>Run a query on Solr and return the results</description> >> + <attribute mode="IN" name="indexName" optional="false" type="String"/> >> <attribute mode="IN" optional="true" name="productCategoryId" type="String"/> >> <attribute mode="IN" optional="true" name="productId" type="String"/> >> <attribute mode="IN" optional="true" name="catalogId" type="String"/> >> @@ -173,6 +173,7 @@ under the License. >> location="org.ofbiz.solr.SolrProductSearch" >> invoke="getSideDeepCategories" debug="true" validate="true"> >> <description>Run a query on Solr and return the results</description> >> + <attribute mode="IN" name="indexName" optional="false" type="String"/> >> <attribute mode="IN" optional="true" name="productCategoryId" type="String"/> >> <attribute mode="IN" optional="true" name="catalogId" type="String"/> >> <attribute name="numFound" type="Long" mode="OUT" optional="false"/> >> >> Modified: ofbiz/trunk/specialpurpose/solr/src/org/ofbiz/solr/SolrProductSearch.java >> URL: http://svn.apache.org/viewvc/ofbiz/trunk/specialpurpose/solr/src/org/ofbiz/solr/SolrProductSearch.java?rev=1730320&r1=1730319&r2=1730320&view=diff >> ============================================================================== >> --- ofbiz/trunk/specialpurpose/solr/src/org/ofbiz/solr/SolrProductSearch.java (original) >> +++ ofbiz/trunk/specialpurpose/solr/src/org/ofbiz/solr/SolrProductSearch.java Sun Feb 14 11:59:59 2016 >> @@ -62,7 +62,6 @@ public abstract class SolrProductSearch >> >> public static final String module = SolrProductSearch.class.getName(); >> >> - >> /** >> * Adds product to solr, with product denoted by productId field in instance attribute >> * - intended for use with ECAs/SECAs. >> @@ -73,6 +72,7 @@ public abstract class SolrProductSearch >> Delegator delegator = dctx.getDelegator(); >> GenericValue productInstance = (GenericValue) context.get("instance"); >> String productId = (String) productInstance.get("productId"); >> + String solrIndexName = (String) context.get("indexName"); >> >> if (SolrUtil.isSolrEcaEnabled()) { >> // Debug.logVerbose("Solr: addToSolr: Running indexing for productId '" + productId + "'", module); >> @@ -80,6 +80,7 @@ public abstract class SolrProductSearch >> GenericValue product = delegator.findOne("Product", UtilMisc.toMap("productId", productId), false); >> Map<String, Object> dispatchContext = ProductUtil.getProductContent(product, dctx, context); >> dispatchContext.put("treatConnectErrorNonFatal", SolrUtil.isEcaTreatConnectErrorNonFatal()); >> + dispatchContext.put("indexName", solrIndexName); >> Map<String, Object> runResult = dispatcher.runSync("addToSolrIndex", dispatchContext); >> String runMsg = ServiceUtil.getErrorMessage(runResult); >> if (UtilValidate.isEmpty(runMsg)) { >> @@ -114,13 +115,14 @@ public abstract class SolrProductSearch >> HttpSolrClient client = null; >> Map<String, Object> result; >> String productId = (String) context.get("productId"); >> + String solrIndexName = (String) context.get("indexName"); >> // connectErrorNonFatal is a necessary option because in some cases it may be considered normal that solr server is unavailable; >> // don't want to return error and abort transactions in these cases. >> Boolean treatConnectErrorNonFatal = (Boolean) context.get("treatConnectErrorNonFatal"); >> try { >> Debug.logInfo("Solr: Generating and indexing document for productId '" + productId + "'", module); >> >> - client = new HttpSolrClient(SolrUtil.solrUrl); >> + client = SolrUtil.getInstance().getHttpSolrClient(solrIndexName); >> //Debug.log(server.ping().toString()); >> >> // Construct Documents >> @@ -185,6 +187,7 @@ public abstract class SolrProductSearch >> * This is faster than reflushing the index each time. >> */ >> public static Map<String, Object> addListToSolrIndex(DispatchContext dctx, Map<String, Object> context) throws GenericEntityException { >> + String solrIndexName = (String) context.get("indexName"); >> HttpSolrClient client = null; >> Map<String, Object> result; >> Boolean treatConnectErrorNonFatal = (Boolean) context.get("treatConnectErrorNonFatal"); >> @@ -204,7 +207,7 @@ public abstract class SolrProductSearch >> docs.add(doc1); >> } >> // push Documents to server >> - client = new HttpSolrClient(SolrUtil.solrUrl); >> + client = SolrUtil.getInstance().getHttpSolrClient(solrIndexName); >> client.add(docs); >> client.commit(); >> >> @@ -258,9 +261,10 @@ public abstract class SolrProductSearch >> public static Map<String, Object> runSolrQuery(DispatchContext dctx, Map<String, Object> context) { >> // get Connection >> HttpSolrClient client = null; >> + String solrIndexName = (String) context.get("indexName"); >> Map<String, Object> result; >> try { >> - client = new HttpSolrClient(SolrUtil.solrUrl); >> + client = SolrUtil.getInstance().getHttpSolrClient(solrIndexName); >> // create Query Object >> SolrQuery solrQuery = new SolrQuery(); >> solrQuery.setQuery((String) context.get("query")); >> @@ -363,6 +367,7 @@ public abstract class SolrProductSearch >> public static Map<String, Object> productsSearch(DispatchContext dctx, Map<String, Object> context) { >> Map<String, Object> result; >> LocalDispatcher dispatcher = dctx.getDispatcher(); >> + String solrIndexName = (String) context.get("indexName"); >> >> try { >> Map<String, Object> dispatchMap = new HashMap<String, Object>(); >> @@ -381,6 +386,7 @@ public abstract class SolrProductSearch >> dispatchMap.put("facet", false); >> dispatchMap.put("spellcheck", true); >> dispatchMap.put("highlight", true); >> + dispatchMap.put("indexName", solrIndexName); >> Map<String, Object> searchResult = dispatcher.runSync("runSolrQuery", dispatchMap); >> QueryResponse queryResult = (QueryResponse) searchResult.get("queryResult"); >> result = ServiceUtil.returnSuccess(); >> @@ -403,6 +409,7 @@ public abstract class SolrProductSearch >> public static Map<String, Object> keywordSearch(DispatchContext dctx, Map<String, Object> context) { >> Map<String, Object> result; >> LocalDispatcher dispatcher = dctx.getDispatcher(); >> + String solrIndexName = (String) context.get("indexName"); >> >> try { >> if (context.get("query") == null || context.get("query").equals("")) >> @@ -418,6 +425,8 @@ public abstract class SolrProductSearch >> if (context.get("queryFilter") != null) >> dispatchMap.put("queryFilter", context.get("queryFilter")); >> dispatchMap.put("spellcheck", true); >> + dispatchMap.put("indexName", solrIndexName); >> + >> Map<String, Object> searchResult = dispatcher.runSync("runSolrQuery", dispatchMap); >> QueryResponse queryResult = (QueryResponse) searchResult.get("queryResult"); >> >> @@ -479,6 +488,7 @@ public abstract class SolrProductSearch >> */ >> public static Map<String, Object> getAvailableCategories(DispatchContext dctx, Map<String, Object> context) { >> Map<String, Object> result; >> + String solrIndexName = (String) context.get("indexName"); >> try { >> boolean displayProducts = false; >> if (UtilValidate.isNotEmpty(context.get("displayProducts"))) >> @@ -494,10 +504,9 @@ public abstract class SolrProductSearch >> if (UtilValidate.isNotEmpty(context.get("catalogId"))) >> catalogId = (String) context.get("catalogId"); >> >> - //String productCategoryId = (String) context.get("productCategoryId") != null ? CategoryUtil.getCategoryNameWithTrail((String) context.get("productCategoryId"), dctx): null; >> String productCategoryId = (String) context.get("productCategoryId") != null ? CategoryUtil.getCategoryNameWithTrail((String) context.get("productCategoryId"),dctx) : null; >> - Debug.logInfo("productCategoryId "+productCategoryId, module); >> - Map<String, Object> query = SolrUtil.categoriesAvailable(catalogId, productCategoryId, (String) context.get("productId"), displayProducts, viewIndex, viewSize); >> + Debug.logInfo("productCategoryId " + productCategoryId, module); >> + Map<String, Object> query = SolrUtil.categoriesAvailable(catalogId, productCategoryId, (String) context.get("productId"), displayProducts, viewIndex, viewSize, solrIndexName); >> >> QueryResponse cat = (QueryResponse) query.get("rows"); >> result = ServiceUtil.returnSuccess(); >> @@ -508,7 +517,7 @@ public abstract class SolrProductSearch >> FacetField field = (FacetField) catIterator.next(); >> List<Count> catL = (List<Count>) field.getValues(); >> if (catL != null) { >> - // log.info("FacetFields = "+catL); >> + // Debug.logInfo("FacetFields = " + catL, module); >> for (Iterator<Count> catIter = catL.iterator(); catIter.hasNext();) { >> FacetField.Count f = (FacetField.Count) catIter.next(); >> if (f.getCount() > 0) { >> @@ -517,7 +526,7 @@ public abstract class SolrProductSearch >> } >> result.put("categories", categories); >> result.put("numFound", cat.getResults().getNumFound()); >> - // log.info("The returned map is this:"+result); >> + // Debug.logInfo("The returned map is this:" + result, module); >> } >> } >> } catch (Exception e) { >> @@ -533,6 +542,7 @@ public abstract class SolrProductSearch >> */ >> public static Map<String, Object> getSideDeepCategories(DispatchContext dctx, Map<String, Object> context) { >> Map<String, Object> result; >> + String solrIndexName = (String) context.get("indexName"); >> try { >> String catalogId = null; >> if (UtilValidate.isNotEmpty(context.get("catalogId"))) >> @@ -556,7 +566,7 @@ public abstract class SolrProductSearch >> int level = Integer.parseInt(categoryPathArray[0]); >> String facetQuery = CategoryUtil.getFacetFilterForCategory(categoryPath, dctx); >> //Debug.logInfo("categoryPath: "+categoryPath + " facetQuery: "+facetQuery,module); >> - Map<String, Object> query = SolrUtil.categoriesAvailable(catalogId, categoryPath, null, facetQuery,false, 0, 0); >> + Map<String, Object> query = SolrUtil.categoriesAvailable(catalogId, categoryPath, null, facetQuery, false, 0, 0, solrIndexName); >> QueryResponse cat = (QueryResponse) query.get("rows"); >> List<Map<String, Object>> categories = new ArrayList<Map<String, Object>>(); >> >> @@ -612,12 +622,13 @@ public abstract class SolrProductSearch >> GenericDelegator delegator = (GenericDelegator) dctx.getDelegator(); >> LocalDispatcher dispatcher = dctx.getDispatcher(); >> GenericValue userLogin = (GenericValue) context.get("userLogin"); >> - Locale locale = new Locale("de_DE"); >> + Locale locale = (Locale) context.get("locale"); >> + String solrIndexName = (String) context.get("indexName"); >> >> Boolean treatConnectErrorNonFatal = (Boolean) context.get("treatConnectErrorNonFatal"); >> >> try { >> - client = new HttpSolrClient(SolrUtil.solrUrl); >> + client = SolrUtil.getInstance().getHttpSolrClient(solrIndexName); >> >> // now lets fetch all products >> List<Map<String, Object>> solrDocs = new ArrayList<Map<String, Object>>(); >> @@ -642,7 +653,7 @@ public abstract class SolrProductSearch >> >> // THis adds all products to the Index (instantly) >> Map<String, Object> runResult = dispatcher.runSync("addListToSolrIndex", UtilMisc.toMap("fieldList", solrDocs, "userLogin", userLogin, >> - "locale", locale, "treatConnectErrorNonFatal", treatConnectErrorNonFatal)); >> + "locale", locale, "indexName", solrIndexName, "treatConnectErrorNonFatal", treatConnectErrorNonFatal)); >> >> String runMsg = ServiceUtil.getErrorMessage(runResult); >> if (UtilValidate.isEmpty(runMsg)) { >> >> Modified: ofbiz/trunk/specialpurpose/solr/src/org/ofbiz/solr/SolrUtil.java >> URL: http://svn.apache.org/viewvc/ofbiz/trunk/specialpurpose/solr/src/org/ofbiz/solr/SolrUtil.java?rev=1730320&r1=1730319&r2=1730320&view=diff >> ============================================================================== >> --- ofbiz/trunk/specialpurpose/solr/src/org/ofbiz/solr/SolrUtil.java (original) >> +++ ofbiz/trunk/specialpurpose/solr/src/org/ofbiz/solr/SolrUtil.java Sun Feb 14 11:59:59 2016 >> @@ -18,12 +18,26 @@ >> *******************************************************************************/ >> package org.ofbiz.solr; >> >> +import java.io.IOException; >> import java.util.HashMap; >> import java.util.Iterator; >> import java.util.List; >> import java.util.Map; >> import java.util.Set; >> >> +import javax.net.ssl.SSLContext; >> + >> +import org.apache.http.client.ClientProtocolException; >> +import org.apache.http.client.config.RequestConfig; >> +import org.apache.http.client.methods.CloseableHttpResponse; >> +import org.apache.http.client.methods.HttpGet; >> +import org.apache.http.client.protocol.HttpClientContext; >> +import org.apache.http.conn.ssl.NoopHostnameVerifier; >> +import org.apache.http.conn.ssl.SSLConnectionSocketFactory; >> +import org.apache.http.ssl.SSLContexts; >> +import org.apache.http.conn.ssl.TrustSelfSignedStrategy; >> +import org.apache.http.impl.client.CloseableHttpClient; >> +import org.apache.http.impl.client.HttpClients; >> import org.apache.solr.client.solrj.SolrQuery; >> import org.apache.solr.client.solrj.SolrRequest.METHOD; >> import org.apache.solr.client.solrj.impl.HttpSolrClient; >> @@ -33,6 +47,7 @@ import org.ofbiz.base.component.Componen >> import org.ofbiz.base.component.ComponentConfig.WebappInfo; >> import org.ofbiz.base.component.ComponentException; >> import org.ofbiz.base.util.Debug; >> +import org.ofbiz.base.util.FileUtil; >> import org.ofbiz.base.util.UtilGenerics; >> import org.ofbiz.base.util.UtilProperties; >> import org.ofbiz.base.util.UtilValidate; >> @@ -41,7 +56,7 @@ import org.ofbiz.entity.GenericEntityExc >> /** >> * Solr utility class. >> */ >> -public abstract class SolrUtil { >> +public final class SolrUtil { >> >> public static final String module = SolrUtil.class.getName(); >> private static String[] solrProdAttribute = { "productId", "internalName", "manu", "size", "smallImage", "mediumImage", "largeImage", "listPrice", "defaultPrice", "inStock", "isVirtual" }; >> @@ -49,7 +64,27 @@ public abstract class SolrUtil { >> public static final String solrConfigName = "solrconfig.properties"; >> public static final String solrUrl = makeSolrWebappUrl(); >> >> - public static String makeSolrWebappUrl() { >> + protected static final String socketTimeoutString = UtilProperties.getPropertyValue(solrConfigName, "solr.client.socket.timeout"); >> + >> + protected static final String connectionTimeoutString = UtilProperties.getPropertyValue(solrConfigName, "solr.client.connection.timeout"); >> + >> + protected static final String clientUsername = UtilProperties.getPropertyValue(solrConfigName, "solr.client.username"); >> + >> + protected static final String clientPassword = UtilProperties.getPropertyValue(solrConfigName, "solr.client.password"); >> + >> + protected static final Integer socketTimeout = getSocketTimeout(); >> + >> + protected static final Integer connectionTimeout = getConnectionTimeout(); >> + >> + protected static final String trustSelfSignedCertString = UtilProperties.getPropertyValue(solrConfigName, "solr.client.trust.selfsigned.cert", "false"); >> + >> + protected static final boolean trustSelfSignedCert = getTrustSelfSignedCert(); >> + >> + protected SolrUtil() { >> + // empty constructor >> + } >> + >> + public static String makeSolrWebappUrl() { >> final String solrWebappProtocol = UtilProperties.getPropertyValue(solrConfigName, "solr.webapp.protocol"); >> final String solrWebappDomainName = UtilProperties.getPropertyValue(solrConfigName, "solr.webapp.domainName"); >> final String solrWebappPath = UtilProperties.getPropertyValue(solrConfigName, "solr.webapp.path"); >> @@ -58,14 +93,42 @@ public abstract class SolrUtil { >> String solrPort; >> if (UtilValidate.isNotEmpty(solrWebappPortOverride)) { >> solrPort = solrWebappPortOverride; >> - } >> - else { >> + } else { >> solrPort = UtilProperties.getPropertyValue("url", ("https".equals(solrWebappProtocol) ? "port.https" : "port.http")); >> } >> >> return solrWebappProtocol + "://" + solrWebappDomainName + ":" + solrPort + solrWebappPath; >> } >> >> + private static Integer getSocketTimeout() { >> + if (UtilValidate.isNotEmpty(socketTimeoutString)) { >> + try { >> + return Integer.parseInt(socketTimeoutString); >> + } catch (Exception e) { >> + return null; >> + } >> + } >> + return null; >> + } >> + >> + private static Integer getConnectionTimeout() { >> + if (UtilValidate.isNotEmpty(connectionTimeoutString)) { >> + try { >> + return Integer.parseInt(connectionTimeoutString); >> + } catch (Exception e) { >> + return null; >> + } >> + } >> + return null; >> + } >> + >> + private static boolean getTrustSelfSignedCert() { >> + if ("true".equals(trustSelfSignedCertString)) { >> + return true; >> + } >> + return false; >> + } >> + >> public static boolean isSolrEcaEnabled() { >> Boolean ecaEnabled = null; >> String sysProp = System.getProperty("ofbiz.solr.eca.enabled"); >> @@ -181,18 +244,18 @@ public abstract class SolrUtil { >> return doc1; >> } >> >> - public static Map<String, Object> categoriesAvailable(String catalogId, String categoryId, String productId, boolean displayproducts, int viewIndex, int viewSize) { >> - return categoriesAvailable(catalogId,categoryId,productId,null,displayproducts,viewIndex,viewSize); >> + public static Map<String, Object> categoriesAvailable(String catalogId, String categoryId, String productId, boolean displayproducts, int viewIndex, int viewSize, String solrIndexName) { >> + return categoriesAvailable(catalogId, categoryId, productId, null, displayproducts, viewIndex, viewSize, solrIndexName); >> } >> >> - public static Map<String, Object> categoriesAvailable(String catalogId, String categoryId, String productId, String facetPrefix, boolean displayproducts, int viewIndex, int viewSize) { >> + public static Map<String, Object> categoriesAvailable(String catalogId, String categoryId, String productId, String facetPrefix, boolean displayproducts, int viewIndex, int viewSize, String solrIndexName) { >> // create the data model >> Map<String, Object> result = new HashMap<String, Object>(); >> HttpSolrClient client = null; >> QueryResponse returnMap = new QueryResponse(); >> try { >> // do the basic query >> - client = new HttpSolrClient(solrUrl); >> + client = getInstance().getHttpSolrClient(solrIndexName); >> // create Query Object >> String query = "inStock[1 TO *]"; >> if (categoryId != null) >> @@ -235,4 +298,68 @@ public abstract class SolrUtil { >> return result; >> } >> >> + private CloseableHttpClient getAllowAllHttpClient() { >> + try { >> + // Trust own CA and all self-signed certs >> + SSLContext sslContext = SSLContexts.custom() >> + .loadTrustMaterial(FileUtil.getFile("component://base/config/ofbizssl.jks"), "changeit".toCharArray(), >> + new TrustSelfSignedStrategy()) >> + .build(); >> + // No host name verifier >> + SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory( >> + sslContext, >> + NoopHostnameVerifier.INSTANCE); >> + CloseableHttpClient httpClient = HttpClients.custom() >> + .setSSLSocketFactory(sslsf) >> + .build(); >> + return httpClient; >> + } catch (Exception e) { >> + return HttpClients.createDefault(); >> + } >> + } >> + >> + public static SolrUtil getInstance() { >> + return new SolrUtil(); >> + } >> + >> + public HttpSolrClient getHttpSolrClient(String solrIndexName) throws ClientProtocolException, IOException { >> + HttpClientContext httpContext = HttpClientContext.create(); >> + >> + CloseableHttpClient httpClient = null; >> + if (trustSelfSignedCert) { >> + httpClient = getAllowAllHttpClient(); >> + } else { >> + httpClient = HttpClients.createDefault(); >> + } >> + >> + RequestConfig requestConfig = null; >> + if (UtilValidate.isNotEmpty(socketTimeout) && UtilValidate.isNotEmpty(connectionTimeout)) { >> + requestConfig = RequestConfig.custom() >> + .setSocketTimeout(socketTimeout) >> + .setConnectTimeout(connectionTimeout) >> + .setRedirectsEnabled(true) >> + .build(); >> + } else if (UtilValidate.isNotEmpty(socketTimeout)) { >> + requestConfig = RequestConfig.custom() >> + .setSocketTimeout(socketTimeout) >> + .setRedirectsEnabled(true) >> + .build(); >> + } else if (UtilValidate.isNotEmpty(connectionTimeout)) { >> + requestConfig = RequestConfig.custom() >> + .setConnectTimeout(connectionTimeout) >> + .setRedirectsEnabled(true) >> + .build(); >> + } else { >> + requestConfig = RequestConfig.custom() >> + .setRedirectsEnabled(true) >> + .build(); >> + } >> + >> + HttpGet httpLogin = new HttpGet(solrUrl + "/control/login?USERNAME=" + clientUsername + "&PASSWORD=" + clientPassword); >> + httpLogin.setConfig(requestConfig); >> + CloseableHttpResponse loginResponse = httpClient.execute(httpLogin, httpContext); >> + loginResponse.close(); >> + return new HttpSolrClient(solrUrl + "/" + solrIndexName, httpClient); >> + } >> + >> } >> >> >> >> > > > > |
Administrator
|
Sorry to push you a bit more but what about https://issues.apache.org/jira/browse/OFBIZ-6755 ?
New features w/o security are not the most wanted ;) Thanks Jacques Le 17/02/2016 17:57, Jacques Le Roux a écrit : > Sorry Jinghai, > > I could not wait, I need to have the tests clean to move ahead. So I reverted r1730320 at r1730873 > > Jacques > > Le 15/02/2016 03:51, Shi Jinghai a écrit : >> Thank you Jacques! >> >> I'll check the problem you mentioned. >> >> -----邮件原件----- >> 发件人: Jacques Le Roux [mailto:[hidden email]] >> 发送时间: 2016年2月14日 22:12 >> 收件人: [hidden email] >> 主题: Re: svn commit: r1730320 - in /ofbiz/trunk/specialpurpose/solr: config/ home/solrdefault/conf/ lib/runtime/ servicedef/ src/org/ofbiz/solr/ >> webapp/solr/WEB-INF/lib/ >> >> Sorry Jinghai >> >> With this commit a lot of tests don't pass >> Must be related with the httpmime-4.4.1.jar change or a dependency on solr component, I did not check |
Free forum by Nabble | Edit this page |