OFBiz › OFBiz - User

How to import a pre existing product database

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

26 messages Options

James Lawton

How to import a pre existing product database

How would I import a pre existing product database? I currently have it in a
CSV and I am looking for a method of getting that information into the
database so that the imported items will show up in the catalog

Thanks

James

BJ Freeman

Re: How to import a pre existing product database

questions like this you can use google like
ofbiz import csv
https://cwiki.apache.org/confluence/display/OFBIZ/Handling+of+External+data
look at
Use the OFBiz's Data File Tools to read you Cvs in.

=========================
BJ Freeman
Strategic Power Office with Supplier Automation <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
Specialtymarket.com <http://www.specialtymarket.com/>
Systems Integrator-- Glad to Assist

Chat Y! messenger: bjfr33man

James Lawton sent the following on 11/9/2010 8:34 AM:

> How would I import a pre existing product database? I currently have it in a
> CSV and I am looking for a method of getting that information into the
> database so that the imported items will show up in the catalog
>
>
>
> Thanks
>
> James
>
>

James Lawton

RE: How to import a pre existing product database

Thanks - I'll dig through this and see if I can get it to work

James

-----Original Message-----
From: BJ Freeman [mailto:[hidden email]]
Sent: November-09-10 12:24 PM
To: [hidden email]
Subject: Re: How to import a pre existing product database

questions like this you can use google like ofbiz import csv
https://cwiki.apache.org/confluence/display/OFBIZ/Handling+of+External+data
look at
Use the OFBiz's Data File Tools to read you Cvs in.

=========================
BJ Freeman
Strategic Power Office with Supplier Automation
<http://www.businessesnetwork.com/automation/viewforum.php?f=52>
Specialtymarket.com <http://www.specialtymarket.com/> Systems Integrator--
Glad to Assist

Chat Y! messenger: bjfr33man

James Lawton sent the following on 11/9/2010 8:34 AM:

> How would I import a pre existing product database? I currently have
> it in a CSV and I am looking for a method of getting that information
> into the database so that the imported items will show up in the
> catalog
>
>
>
> Thanks
>
> James
>
>

James McGill-5

Re: How to import a pre existing product database

In reply to this post by James Lawton

On Tue, Nov 9, 2010 at 9:34 AM, James Lawton <[hidden email]>wrote:

> How would I import a pre existing product database? I currently have it in
> a
> CSV and I am looking for a method of getting that information into the
> database so that the imported items will show up in the catalog
>

OFBiz ETL is best done as XML seed data.
In your scenario I would seriously consider converting the CSV into XML (a
job for Perl),
and perhaps doing some input validation as part of that conversion, and then
loading the XML
as seed data.

If you really have a lot of ETL to do (e.g., processing that will take hours
or days to run through the service interface) you might be better off going
straight to the database. You lose some of the traceability this way (e.g.,
logging and event side-effects) so it isn't recommended. (We ran into
problems when loading in several years worth of historical data, orders and
inventory transactions.)

--
James McGill
Phoenix AZ

James Lawton

RE: How to import a pre existing product database

Hmmm, I have about 22,000 new products in the small demo sampling that we want to try to upload into the Ofbiz database. How long would that take to load through the service interfaces?

Like you we also want to upload a bunch of historical data, orders, etc...

What were the issues that you ran into? How did you get around it?

Thanks
James

-----Original Message-----
From: James McGill [mailto:[hidden email]]
Sent: November-09-10 3:38 PM
To: [hidden email]
Subject: Re: How to import a pre existing product database

On Tue, Nov 9, 2010 at 9:34 AM, James Lawton <[hidden email]>wrote:

> How would I import a pre existing product database? I currently have
> it in a CSV and I am looking for a method of getting that information
> into the database so that the imported items will show up in the
> catalog
>

OFBiz ETL is best done as XML seed data.
In your scenario I would seriously consider converting the CSV into XML (a job for Perl), and perhaps doing some input validation as part of that conversion, and then loading the XML as seed data.

If you really have a lot of ETL to do (e.g., processing that will take hours or days to run through the service interface) you might be better off going straight to the database. You lose some of the traceability this way (e.g., logging and event side-effects) so it isn't recommended. (We ran into problems when loading in several years worth of historical data, orders and inventory transactions.)

--
James McGill
Phoenix AZ

BJ Freeman

Re: How to import a pre existing product database

In reply to this post by James McGill-5

all web tools imports into memory so is limited to the amount of free
memory of the sever.

I imports files that are gigs in size so read in the files. I typical
file has 25 columns and takes about 15min.
There is no need to write code to convert files to xml since the
https://cwiki.apache.org/confluence/display/OFBENDUSER/OFBiz%27s+Data+File+Tools

=========================
BJ Freeman
Strategic Power Office with Supplier Automation <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
Specialtymarket.com <http://www.specialtymarket.com/>
Systems Integrator-- Glad to Assist

Chat Y! messenger: bjfr33man
James McGill sent the following on 11/9/2010 12:38 PM:

> On Tue, Nov 9, 2010 at 9:34 AM, James Lawton<[hidden email]>wrote:
>
>> How would I import a pre existing product database? I currently have it in
>> a
>> CSV and I am looking for a method of getting that information into the
>> database so that the imported items will show up in the catalog
>>
>
>
> OFBiz ETL is best done as XML seed data.
> In your scenario I would seriously consider converting the CSV into XML (a
> job for Perl),
> and perhaps doing some input validation as part of that conversion, and then
> loading the XML
> as seed data.
>
> If you really have a lot of ETL to do (e.g., processing that will take hours
> or days to run through the service interface) you might be better off going
> straight to the database. You lose some of the traceability this way (e.g.,
> logging and event side-effects) so it isn't recommended. (We ran into
> problems when loading in several years worth of historical data, orders and
> inventory transactions.)
>
> --
> James McGill
> Phoenix AZ
>

James Lawton

RE: How to import a pre existing product database

I am probably doing something stupid here...I have dug into the link that was posted and it looks simple enough...however the code snippet is for a fixed width file rather than a comma delimited file...and my attempts to create the right schema for importing product are failing...If someone has a CSV file snippet to share that would be awesome

So heading in a different direction I created the entity data snippet below:


<entity-engine-xml>

<DataImportProduct productId="YG0081ATF" productTypeId="FINISHED_GOOD" description="1/8 x 1-59/64 CARBIDE DREAM DRILL W/O COOLANT HOLES (3XD)" price="22.13" priceCurrencyUomId="USD" supplierPartyId="YG1" purchasePrice=""/>
<DataImportProduct productId="YG0081BTF" productTypeId="FINISHED_GOOD" description="1/8 x 15/64 CARBIDE DREAM DRILL W/ COOLANT HOLE" price="105.73" priceCurrencyUomId="USD" supplierPartyId="YG1" purchasePrice=""/>

</entity-engine-xml>

However every time I have tried to use the XML IMport tool it just keeps coming back and saying that the either URL / file or the XML are not complete

What am I doing wrong?

-----Original Message-----
From: BJ Freeman [mailto:[hidden email]]
Sent: November-09-10 6:00 PM
To: [hidden email]
Subject: Re: How to import a pre existing product database

all web tools imports into memory so is limited to the amount of free memory of the sever.

I imports files that are gigs in size so read in the files. I typical file has 25 columns and takes about 15min.
There is no need to write code to convert files to xml since the https://cwiki.apache.org/confluence/display/OFBENDUSER/OFBiz%27s+Data+File+Tools

=========================
BJ Freeman
Strategic Power Office with Supplier Automation <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
Specialtymarket.com <http://www.specialtymarket.com/>
Systems Integrator-- Glad to Assist

Chat Y! messenger: bjfr33man
James McGill sent the following on 11/9/2010 12:38 PM:

BJ Freeman

Re: How to import a pre existing product database

I am not clear what you are doing.
the example you show is for an entity DataImportProduct
which does not exist in the OOTB ofbiz
take a look at

https://issues.apache.org/jira/browse/OFBIZ-2534
for tab examples.
also
https://issues.apache.org/jira/browse/OFBIZ-2518

=========================

BJ Freeman
Strategic Power Office with Supplier Automation <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
Specialtymarket.com <http://www.specialtymarket.com/>
Systems Integrator-- Glad to Assist

Chat Y! messenger: bjfr33man
James Lawton sent the following on 11/25/2010 6:36 PM:

> I am probably doing something stupid here...I have dug into the link that was posted and it looks simple enough...however the code snippet is for a fixed width file rather than a comma delimited file...and my attempts to create the right schema for importing product are failing...If someone has a CSV file snippet to share that would be awesome
>
> So heading in a different direction I created the entity data snippet below:
>
> 
> <entity-engine-xml>
>
> <DataImportProduct productId="YG0081ATF" productTypeId="FINISHED_GOOD" description="1/8 x 1-59/64 CARBIDE DREAM DRILL W/O COOLANT HOLES (3XD)" price="22.13" priceCurrencyUomId="USD" supplierPartyId="YG1" purchasePrice=""/>
> <DataImportProduct productId="YG0081BTF" productTypeId="FINISHED_GOOD" description="1/8 x 15/64 CARBIDE DREAM DRILL W/ COOLANT HOLE" price="105.73" priceCurrencyUomId="USD" supplierPartyId="YG1" purchasePrice=""/>
>
>
> </entity-engine-xml>
>
> However every time I have tried to use the XML IMport tool it just keeps coming back and saying that the either URL / file or the XML are not complete
>
> What am I doing wrong?
>
> -----Original Message-----
> From: BJ Freeman [mailto:[hidden email]]
> Sent: November-09-10 6:00 PM
> To: [hidden email]
> Subject: Re: How to import a pre existing product database
>
> all web tools imports into memory so is limited to the amount of free memory of the sever.
>
> I imports files that are gigs in size so read in the files. I typical file has 25 columns and takes about 15min.
> There is no need to write code to convert files to xml since the https://cwiki.apache.org/confluence/display/OFBENDUSER/OFBiz%27s+Data+File+Tools
>
>
>
>
>
> =========================
> BJ Freeman
> Strategic Power Office with Supplier Automation<http://www.businessesnetwork.com/automation/viewforum.php?f=52>
> Specialtymarket.com<http://www.specialtymarket.com/>
> Systems Integrator-- Glad to Assist
>
> Chat Y! messenger: bjfr33man
> James McGill sent the following on 11/9/2010 12:38 PM:
>
>> On Tue, Nov 9, 2010 at 9:34 AM, James Lawton<[hidden email]>wrote:
>>
>>> How would I import a pre existing product database? I currently have it in
>>> a
>>> CSV and I am looking for a method of getting that information into the
>>> database so that the imported items will show up in the catalog
>>>
>>
>>
>> OFBiz ETL is best done as XML seed data.
>> In your scenario I would seriously consider converting the CSV into XML (a
>> job for Perl),
>> and perhaps doing some input validation as part of that conversion, and then
>> loading the XML
>> as seed data.
>>
>> If you really have a lot of ETL to do (e.g., processing that will take hours
>> or days to run through the service interface) you might be better off going
>> straight to the database. You lose some of the traceability this way (e.g.,
>> logging and event side-effects) so it isn't recommended. (We ran into
>> problems when loading in several years worth of historical data, orders and
>> inventory transactions.)
>>
>> --
>> James McGill
>> Phoenix AZ
>>
>
>
>

James Lawton

RE: How to import a pre existing product database

I think I am getting closer - thanks for the tips and the patience!

I have been through all the files and here is what I have gotten when running my definition file through the data tools

org.ofbiz.widget.screen.ScreenRenderException: Error rendering screen [component://webtools/widget/MiscScreens.xml#viewdatafile]: groovy.lang.MissingPropertyException: No such property: definitionUrl for class: viewdatafile (No such property: definitionUrl for class: viewdatafile)

Here is the xsd file I was running:

<data-file name="itemdatabasereport.txt" separator-style="delimited" type-code="UTF-8"
delimiter="&#009" text-delimiter=""
description="this takes the product data from jeffs file"
has-header="true">
<record name="Product">
<field name="manufacturer_party_id" ignored="false" type="String"/>
<field name="productId" ignored="false" type="String"/>
<field name="primaryProductCategoryId" expression="true"
default-value="tools" type="String"/>
<field name="productTypeId" expression="true"
default-value="FINISHED_GOOD" type="String"/>
<field name="productName" ignored="false" type="String"/>
<field name="description" ignored="false" type="String"/>
<field name="quantityIncluded" expression="true" default-value="1"
type="String"/>
<field name="price" ignored="true" type="String"/>
</record>
</data-file>

What's wrong with my definition file which is causing the data tool to throw up?

Thanks
James

-----Original Message-----
From: BJ Freeman [mailto:[hidden email]]
Sent: November-26-10 1:36 AM
To: [hidden email]
Subject: Re: How to import a pre existing product database

I am not clear what you are doing.
the example you show is for an entity DataImportProduct which does not exist in the OOTB ofbiz take a look at

https://issues.apache.org/jira/browse/OFBIZ-2534
for tab examples.
also
https://issues.apache.org/jira/browse/OFBIZ-2518

=========================

BJ Freeman
Strategic Power Office with Supplier Automation <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
Specialtymarket.com <http://www.specialtymarket.com/> Systems Integrator-- Glad to Assist

Chat Y! messenger: bjfr33man
James Lawton sent the following on 11/25/2010 6:36 PM:

> I am probably doing something stupid here...I have dug into the link
> that was posted and it looks simple enough...however the code snippet
> is for a fixed width file rather than a comma delimited file...and my
> attempts to create the right schema for importing product are
> failing...If someone has a CSV file snippet to share that would be
> awesome
>
> So heading in a different direction I created the entity data snippet below:
>
>  <entity-engine-xml>
>
> <DataImportProduct productId="YG0081ATF" productTypeId="FINISHED_GOOD" description="1/8 x 1-59/64 CARBIDE DREAM DRILL W/O COOLANT HOLES (3XD)" price="22.13" priceCurrencyUomId="USD" supplierPartyId="YG1" purchasePrice=""/>
> <DataImportProduct productId="YG0081BTF"
> productTypeId="FINISHED_GOOD" description="1/8 x 15/64 CARBIDE DREAM
> DRILL W/ COOLANT HOLE" price="105.73" priceCurrencyUomId="USD"
> supplierPartyId="YG1" purchasePrice=""/>
>
>
> </entity-engine-xml>
>
> However every time I have tried to use the XML IMport tool it just
> keeps coming back and saying that the either URL / file or the XML are
> not complete
>
> What am I doing wrong?
>
> -----Original Message-----
> From: BJ Freeman [mailto:[hidden email]]
> Sent: November-09-10 6:00 PM
> To: [hidden email]
> Subject: Re: How to import a pre existing product database
>
> all web tools imports into memory so is limited to the amount of free memory of the sever.
>
> I imports files that are gigs in size so read in the files. I typical file has 25 columns and takes about 15min.
> There is no need to write code to convert files to xml since the
> https://cwiki.apache.org/confluence/display/OFBENDUSER/OFBiz%27s+Data+
> File+Tools
>
>
>
>
>
> =========================
> BJ Freeman
> Strategic Power Office with Supplier
> Automation<http://www.businessesnetwork.com/automation/viewforum.php?f
> =52> Specialtymarket.com<http://www.specialtymarket.com/>
> Systems Integrator-- Glad to Assist
>
> Chat Y! messenger: bjfr33man
> James McGill sent the following on 11/9/2010 12:38 PM:
>
>> On Tue, Nov 9, 2010 at 9:34 AM, James Lawton<[hidden email]>wrote:
>>
>>> How would I import a pre existing product database? I currently have
>>> it in a CSV and I am looking for a method of getting that
>>> information into the database so that the imported items will show
>>> up in the catalog
>>>
>>
>>
>> OFBiz ETL is best done as XML seed data.
>> In your scenario I would seriously consider converting the CSV into
>> XML (a job for Perl), and perhaps doing some input validation as part
>> of that conversion, and then loading the XML as seed data.
>>
>> If you really have a lot of ETL to do (e.g., processing that will
>> take hours or days to run through the service interface) you might be
>> better off going straight to the database. You lose some of the
>> traceability this way (e.g., logging and event side-effects) so it
>> isn't recommended. (We ran into problems when loading in several
>> years worth of historical data, orders and inventory transactions.)
>>
>> --
>> James McGill
>> Phoenix AZ
>>
>
>
>

BJ Freeman

Re: How to import a pre existing product database

there should be more to the log that would pinpoint what is going on.
I am guessing it is your input to the form, since this has not been
changed 09.
if you look at the file mentioned you will see
<script
location="component://webtools/webapp/webtools/WEB-INF/actions/datafile/viewdatafile.groovy"/>
that is where the error referenced.

your xml(not xsd) should have field names as you see them in webtools.
when you have underscores like manufacturer_party_id then that is what
is in the db but ofbiz will not recognize it in Webtools or code.

James Lawton sent the following on 11/30/2010 7:34 PM:

> I think I am getting closer - thanks for the tips and the patience!
>
> I have been through all the files and here is what I have gotten when running my definition file through the data tools
>
> org.ofbiz.widget.screen.ScreenRenderException: Error rendering screen [component://webtools/widget/MiscScreens.xml#viewdatafile]: groovy.lang.MissingPropertyException: No such property: definitionUrl for class: viewdatafile (No such property: definitionUrl for class: viewdatafile)
>
>
> Here is the xsd file I was running:
>
>
> <data-file name="itemdatabasereport.txt" separator-style="delimited" type-code="UTF-8"
> delimiter="&#009" text-delimiter=""
> description="this takes the product data from jeffs file"
> has-header="true">
> <record name="Product">
> <field name="manufacturer_party_id" ignored="false" type="String"/>
> <field name="productId" ignored="false" type="String"/>
> <field name="primaryProductCategoryId" expression="true"
> default-value="tools" type="String"/>
> <field name="productTypeId" expression="true"
> default-value="FINISHED_GOOD" type="String"/>
> <field name="productName" ignored="false" type="String"/>
> <field name="description" ignored="false" type="String"/>
> <field name="quantityIncluded" expression="true" default-value="1"
> type="String"/>
> <field name="price" ignored="true" type="String"/>
> </record>
> </data-file>
>
>
> What's wrong with my definition file which is causing the data tool to throw up?
>
> Thanks
> James
>
>
>
> -----Original Message-----
> From: BJ Freeman [mailto:[hidden email]]
> Sent: November-26-10 1:36 AM
> To: [hidden email]
> Subject: Re: How to import a pre existing product database
>
> I am not clear what you are doing.
> the example you show is for an entity DataImportProduct which does not exist in the OOTB ofbiz take a look at
>
> https://issues.apache.org/jira/browse/OFBIZ-2534
> for tab examples.
> also
> https://issues.apache.org/jira/browse/OFBIZ-2518
>
> =========================
>
> BJ Freeman
> Strategic Power Office with Supplier Automation<http://www.businessesnetwork.com/automation/viewforum.php?f=52>
> Specialtymarket.com<http://www.specialtymarket.com/> Systems Integrator-- Glad to Assist
>
> Chat Y! messenger: bjfr33man
> James Lawton sent the following on 11/25/2010 6:36 PM:
>> I am probably doing something stupid here...I have dug into the link
>> that was posted and it looks simple enough...however the code snippet
>> is for a fixed width file rather than a comma delimited file...and my
>> attempts to create the right schema for importing product are
>> failing...If someone has a CSV file snippet to share that would be
>> awesome
>>
>> So heading in a different direction I created the entity data snippet below:
>>
>>  <entity-engine-xml>
>>
>> <DataImportProduct productId="YG0081ATF" productTypeId="FINISHED_GOOD" description="1/8 x 1-59/64 CARBIDE DREAM DRILL W/O COOLANT HOLES (3XD)" price="22.13" priceCurrencyUomId="USD" supplierPartyId="YG1" purchasePrice=""/>
>> <DataImportProduct productId="YG0081BTF"
>> productTypeId="FINISHED_GOOD" description="1/8 x 15/64 CARBIDE DREAM
>> DRILL W/ COOLANT HOLE" price="105.73" priceCurrencyUomId="USD"
>> supplierPartyId="YG1" purchasePrice=""/>
>>
>>
>> </entity-engine-xml>
>>
>> However every time I have tried to use the XML IMport tool it just
>> keeps coming back and saying that the either URL / file or the XML are
>> not complete
>>
>> What am I doing wrong?
>>
>> -----Original Message-----
>> From: BJ Freeman [mailto:[hidden email]]
>> Sent: November-09-10 6:00 PM
>> To: [hidden email]
>> Subject: Re: How to import a pre existing product database
>>
>> all web tools imports into memory so is limited to the amount of free memory of the sever.
>>
>> I imports files that are gigs in size so read in the files. I typical file has 25 columns and takes about 15min.
>> There is no need to write code to convert files to xml since the
>> https://cwiki.apache.org/confluence/display/OFBENDUSER/OFBiz%27s+Data+
>> File+Tools
>>
>>
>>
>>
>>
>> =========================
>> BJ Freeman
>> Strategic Power Office with Supplier
>> Automation<http://www.businessesnetwork.com/automation/viewforum.php?f
>> =52> Specialtymarket.com<http://www.specialtymarket.com/>
>> Systems Integrator-- Glad to Assist
>>
>> Chat Y! messenger: bjfr33man
>> James McGill sent the following on 11/9/2010 12:38 PM:
>>
>>> On Tue, Nov 9, 2010 at 9:34 AM, James Lawton<[hidden email]>wrote:
>>>
>>>> How would I import a pre existing product database? I currently have
>>>> it in a CSV and I am looking for a method of getting that
>>>> information into the database so that the imported items will show
>>>> up in the catalog
>>>>
>>>
>>>
>>> OFBiz ETL is best done as XML seed data.
>>> In your scenario I would seriously consider converting the CSV into
>>> XML (a job for Perl), and perhaps doing some input validation as part
>>> of that conversion, and then loading the XML as seed data.
>>>
>>> If you really have a lot of ETL to do (e.g., processing that will
>>> take hours or days to run through the service interface) you might be
>>> better off going straight to the database. You lose some of the
>>> traceability this way (e.g., logging and event side-effects) so it
>>> isn't recommended. (We ran into problems when loading in several
>>> years worth of historical data, orders and inventory transactions.)
>>>
>>> --
>>> James McGill
>>> Phoenix AZ
>>>
>>
>>
>>
>
>
>

James Lawton

RE: How to import a pre existing product database

Thanks

I must be missing the spot in web tools it has a complete list of the field names - could you point that out again? For example what is the right field name for the manufacturer / supplier? I did take that out the db....

Thanks
James

-----Original Message-----
From: BJ Freeman [mailto:[hidden email]]
Sent: November-30-10 11:42 PM
To: [hidden email]
Subject: Re: How to import a pre existing product database

there should be more to the log that would pinpoint what is going on.
I am guessing it is your input to the form, since this has not been changed 09.
if you look at the file mentioned you will see
<script
location="component://webtools/webapp/webtools/WEB-INF/actions/datafile/viewdatafile.groovy"/>
that is where the error referenced.

your xml(not xsd) should have field names as you see them in webtools.
when you have underscores like manufacturer_party_id then that is what is in the db but ofbiz will not recognize it in Webtools or code.

James Lawton sent the following on 11/30/2010 7:34 PM:

BJ Freeman

Re: How to import a pre existing product database

if you look at the table the manufacture field is in
then look up the same in the entities
https://demo-trunk.ofbiz.apache.org/webtools/control/entitymaint
Just to clarify
your file has a Record=Product.
this means you will be creating a Product entity file, so only fields in
the product entity can be created.
https://demo-trunk.ofbiz.apache.org/webtools/control/FindGeneric?entityName=Product&find=true&VIEW_SIZE=50&VIEW_INDEX=0
if you have data that goes in different entities then you need to create
a Record for each entity with its fields.

James Lawton sent the following on 12/1/2010 9:16 AM:

=========================
BJ Freeman
Strategic Power Office with Supplier Automation <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
Specialtymarket.com <http://www.specialtymarket.com/>
Systems Integrator-- Glad to Assist

Chat Y! messenger: bjfr33man

> Thanks
>
> I must be missing the spot in web tools it has a complete list of the field names - could you point that out again? For example what is the right field name for the manufacturer / supplier? I did take that out the db....
>
> Thanks
> James
>
> -----Original Message-----
> From: BJ Freeman [mailto:[hidden email]]
> Sent: November-30-10 11:42 PM
> To: [hidden email]
> Subject: Re: How to import a pre existing product database
>
> there should be more to the log that would pinpoint what is going on.
> I am guessing it is your input to the form, since this has not been changed 09.
> if you look at the file mentioned you will see
> <script
> location="component://webtools/webapp/webtools/WEB-INF/actions/datafile/viewdatafile.groovy"/>
> that is where the error referenced.
>
> your xml(not xsd) should have field names as you see them in webtools.
> when you have underscores like manufacturer_party_id then that is what is in the db but ofbiz will not recognize it in Webtools or code.
>
>
>
>
> James Lawton sent the following on 11/30/2010 7:34 PM:
>
>
>> I think I am getting closer - thanks for the tips and the patience!
>>
>> I have been through all the files and here is what I have gotten when running my definition file through the data tools
>>
>> org.ofbiz.widget.screen.ScreenRenderException: Error rendering screen [component://webtools/widget/MiscScreens.xml#viewdatafile]: groovy.lang.MissingPropertyException: No such property: definitionUrl for class: viewdatafile (No such property: definitionUrl for class: viewdatafile)
>>
>>
>> Here is the xsd file I was running:
>>
>>
>> <data-file name="itemdatabasereport.txt" separator-style="delimited" type-code="UTF-8"
>> delimiter="&#009" text-delimiter=""
>> description="this takes the product data from jeffs file"
>> has-header="true">
>> <record name="Product">
>> <field name="manufacturer_party_id" ignored="false" type="String"/>
>> <field name="productId" ignored="false" type="String"/>
>> <field name="primaryProductCategoryId" expression="true"
>> default-value="tools" type="String"/>
>> <field name="productTypeId" expression="true"
>> default-value="FINISHED_GOOD" type="String"/>
>> <field name="productName" ignored="false" type="String"/>
>> <field name="description" ignored="false" type="String"/>
>> <field name="quantityIncluded" expression="true" default-value="1"
>> type="String"/>
>> <field name="price" ignored="true" type="String"/>
>> </record>
>> </data-file>
>>
>>
>> What's wrong with my definition file which is causing the data tool to throw up?
>>
>> Thanks
>> James
>>
>>
>>
>> -----Original Message-----
>> From: BJ Freeman [mailto:[hidden email]]
>> Sent: November-26-10 1:36 AM
>> To: [hidden email]
>> Subject: Re: How to import a pre existing product database
>>
>> I am not clear what you are doing.
>> the example you show is for an entity DataImportProduct which does not exist in the OOTB ofbiz take a look at
>>
>> https://issues.apache.org/jira/browse/OFBIZ-2534
>> for tab examples.
>> also
>> https://issues.apache.org/jira/browse/OFBIZ-2518
>>
>> =========================
>>
>> BJ Freeman
>> Strategic Power Office with Supplier Automation<http://www.businessesnetwork.com/automation/viewforum.php?f=52>
>> Specialtymarket.com<http://www.specialtymarket.com/> Systems Integrator-- Glad to Assist
>>
>> Chat Y! messenger: bjfr33man
>> James Lawton sent the following on 11/25/2010 6:36 PM:
>>> I am probably doing something stupid here...I have dug into the link
>>> that was posted and it looks simple enough...however the code snippet
>>> is for a fixed width file rather than a comma delimited file...and my
>>> attempts to create the right schema for importing product are
>>> failing...If someone has a CSV file snippet to share that would be
>>> awesome
>>>
>>> So heading in a different direction I created the entity data snippet below:
>>>
>>>  <entity-engine-xml>
>>>
>>> <DataImportProduct productId="YG0081ATF" productTypeId="FINISHED_GOOD" description="1/8 x 1-59/64 CARBIDE DREAM DRILL W/O COOLANT HOLES (3XD)" price="22.13" priceCurrencyUomId="USD" supplierPartyId="YG1" purchasePrice=""/>
>>> <DataImportProduct productId="YG0081BTF"
>>> productTypeId="FINISHED_GOOD" description="1/8 x 15/64 CARBIDE DREAM
>>> DRILL W/ COOLANT HOLE" price="105.73" priceCurrencyUomId="USD"
>>> supplierPartyId="YG1" purchasePrice=""/>
>>>
>>>
>>> </entity-engine-xml>
>>>
>>> However every time I have tried to use the XML IMport tool it just
>>> keeps coming back and saying that the either URL / file or the XML are
>>> not complete
>>>
>>> What am I doing wrong?
>>>
>>> -----Original Message-----
>>> From: BJ Freeman [mailto:[hidden email]]
>>> Sent: November-09-10 6:00 PM
>>> To: [hidden email]
>>> Subject: Re: How to import a pre existing product database
>>>
>>> all web tools imports into memory so is limited to the amount of free memory of the sever.
>>>
>>> I imports files that are gigs in size so read in the files. I typical file has 25 columns and takes about 15min.
>>> There is no need to write code to convert files to xml since the
>>> https://cwiki.apache.org/confluence/display/OFBENDUSER/OFBiz%27s+Data+
>>> File+Tools
>>>
>>>
>>>
>>>
>>>
>>> =========================
>>> BJ Freeman
>>> Strategic Power Office with Supplier
>>> Automation<http://www.businessesnetwork.com/automation/viewforum.php?f
>>> =52> Specialtymarket.com<http://www.specialtymarket.com/>
>>> Systems Integrator-- Glad to Assist
>>>
>>> Chat Y! messenger: bjfr33man
>>> James McGill sent the following on 11/9/2010 12:38 PM:
>>>
>>>> On Tue, Nov 9, 2010 at 9:34 AM, James Lawton<[hidden email]>wrote:
>>>>
>>>>> How would I import a pre existing product database? I currently have
>>>>> it in a CSV and I am looking for a method of getting that
>>>>> information into the database so that the imported items will show
>>>>> up in the catalog
>>>>>
>>>>
>>>>
>>>> OFBiz ETL is best done as XML seed data.
>>>> In your scenario I would seriously consider converting the CSV into
>>>> XML (a job for Perl), and perhaps doing some input validation as part
>>>> of that conversion, and then loading the XML as seed data.
>>>>
>>>> If you really have a lot of ETL to do (e.g., processing that will
>>>> take hours or days to run through the service interface) you might be
>>>> better off going straight to the database. You lose some of the
>>>> traceability this way (e.g., logging and event side-effects) so it
>>>> isn't recommended. (We ran into problems when loading in several
>>>> years worth of historical data, orders and inventory transactions.)
>>>>
>>>> --
>>>> James McGill
>>>> Phoenix AZ
>>>>
>>>
>>>
>>>
>>
>>
>>
>
>
>

James Lawton

RE: How to import a pre existing product database

Thanks for the link and the record information...

I have altered the fields to fit the product record field names and tried to use the "work with data files" tool again

I enter the Definition File Name in the screen
"file://locahost/c:\files\tofino\import data\tofinodata.xml" and select the URL check box and then hit Run. The screen flickers but nothing happens. The instructions say that a drop down box will appear with a list of the definitions to pick from...

Thoughts?

Here is the modified definition file:

<data-file name="c:\files\tofino\import data\itemdatabasereportmod.txt" separator-style="delimited" type-code="UTF-8"
delimiter="&#009" text-delimiter=""
description="this takes the product data from jeffs file"
has-header="true">
<record name="Product">
<field name="manufacturerPartyId" ignored="false" type="String"/>
<field name="productId" ignored="false" type="String"/>
<field name="primaryProductCategoryId" expression="true"
default-value="tools" type="String"/>
<field name="productTypeId" expression="true"
default-value="FINISHED_GOOD" type="String"/>
<field name="productName" ignored="false" type="String"/>
<field name="description" ignored="false" type="String"/>
<field name="quantityIncluded" expression="true" default-value="1"
type="String"/>
<field name="priceDetailText" ignored="true" type="String"/>
</record>
</data-file>

Thanks
James
-----Original Message-----
From: BJ Freeman [mailto:[hidden email]]
Sent: December-01-10 12:37 PM
To: [hidden email]
Subject: Re: How to import a pre existing product database

if you look at the table the manufacture field is in then look up the same in the entities https://demo-trunk.ofbiz.apache.org/webtools/control/entitymaint
Just to clarify
your file has a Record=Product.
this means you will be creating a Product entity file, so only fields in the product entity can be created.
https://demo-trunk.ofbiz.apache.org/webtools/control/FindGeneric?entityName=Product&find=true&VIEW_SIZE=50&VIEW_INDEX=0
if you have data that goes in different entities then you need to create a Record for each entity with its fields.

James Lawton sent the following on 12/1/2010 9:16 AM:

=========================
BJ Freeman
Strategic Power Office with Supplier Automation <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
Specialtymarket.com <http://www.specialtymarket.com/> Systems Integrator-- Glad to Assist

Chat Y! messenger: bjfr33man

> Thanks
>
> I must be missing the spot in web tools it has a complete list of the field names - could you point that out again? For example what is the right field name for the manufacturer / supplier? I did take that out the db....
>
> Thanks
> James
>
> -----Original Message-----
> From: BJ Freeman [mailto:[hidden email]]
> Sent: November-30-10 11:42 PM
> To: [hidden email]
> Subject: Re: How to import a pre existing product database
>
> there should be more to the log that would pinpoint what is going on.
> I am guessing it is your input to the form, since this has not been changed 09.
> if you look at the file mentioned you will see
> <script
> location="component://webtools/webapp/webtools/WEB-INF/actions/datafil
> e/viewdatafile.groovy"/>
> that is where the error referenced.
>
> your xml(not xsd) should have field names as you see them in webtools.
> when you have underscores like manufacturer_party_id then that is what is in the db but ofbiz will not recognize it in Webtools or code.
>
>
>
>
> James Lawton sent the following on 11/30/2010 7:34 PM:
>
>
>> I think I am getting closer - thanks for the tips and the patience!
>>
>> I have been through all the files and here is what I have gotten when
>> running my definition file through the data tools
>>
>> org.ofbiz.widget.screen.ScreenRenderException: Error rendering screen
>> [component://webtools/widget/MiscScreens.xml#viewdatafile]:
>> groovy.lang.MissingPropertyException: No such property: definitionUrl
>> for class: viewdatafile (No such property: definitionUrl for class:
>> viewdatafile)
>>
>>
>> Here is the xsd file I was running:
>>
>>
>> <data-file name="itemdatabasereport.txt" separator-style="delimited" type-code="UTF-8"
>> delimiter="&#009" text-delimiter=""
>> description="this takes the product data from jeffs file"
>> has-header="true">
>> <record name="Product">
>> <field name="manufacturer_party_id" ignored="false" type="String"/>
>> <field name="productId" ignored="false" type="String"/>
>> <field name="primaryProductCategoryId" expression="true"
>> default-value="tools" type="String"/>
>> <field name="productTypeId" expression="true"
>> default-value="FINISHED_GOOD" type="String"/>
>> <field name="productName" ignored="false" type="String"/>
>> <field name="description" ignored="false" type="String"/>
>> <field name="quantityIncluded" expression="true" default-value="1"
>> type="String"/>
>> <field name="price" ignored="true" type="String"/>
>> </record>
>> </data-file>
>>
>>
>> What's wrong with my definition file which is causing the data tool to throw up?
>>
>> Thanks
>> James
>>
>>
>>
>> -----Original Message-----
>> From: BJ Freeman [mailto:[hidden email]]
>> Sent: November-26-10 1:36 AM
>> To: [hidden email]
>> Subject: Re: How to import a pre existing product database
>>
>> I am not clear what you are doing.
>> the example you show is for an entity DataImportProduct which does
>> not exist in the OOTB ofbiz take a look at
>>
>> https://issues.apache.org/jira/browse/OFBIZ-2534
>> for tab examples.
>> also
>> https://issues.apache.org/jira/browse/OFBIZ-2518
>>
>> =========================
>>
>> BJ Freeman
>> Strategic Power Office with Supplier Automation<http://www.businessesnetwork.com/automation/viewforum.php?f=52>
>> Specialtymarket.com<http://www.specialtymarket.com/> Systems Integrator-- Glad to Assist
>>
>> Chat Y! messenger: bjfr33man
>> James Lawton sent the following on 11/25/2010 6:36 PM:
>>> I am probably doing something stupid here...I have dug into the link
>>> that was posted and it looks simple enough...however the code
>>> snippet is for a fixed width file rather than a comma delimited
>>> file...and my attempts to create the right schema for importing
>>> product are failing...If someone has a CSV file snippet to share
>>> that would be awesome
>>>
>>> So heading in a different direction I created the entity data snippet below:
>>>
>>>  <entity-engine-xml>
>>>
>>> <DataImportProduct productId="YG0081ATF" productTypeId="FINISHED_GOOD" description="1/8 x 1-59/64 CARBIDE DREAM DRILL W/O COOLANT HOLES (3XD)" price="22.13" priceCurrencyUomId="USD" supplierPartyId="YG1" purchasePrice=""/>
>>> <DataImportProduct productId="YG0081BTF"
>>> productTypeId="FINISHED_GOOD" description="1/8 x 15/64 CARBIDE DREAM
>>> DRILL W/ COOLANT HOLE" price="105.73" priceCurrencyUomId="USD"
>>> supplierPartyId="YG1" purchasePrice=""/>
>>>
>>>
>>> </entity-engine-xml>
>>>
>>> However every time I have tried to use the XML IMport tool it just
>>> keeps coming back and saying that the either URL / file or the XML
>>> are not complete
>>>
>>> What am I doing wrong?
>>>
>>> -----Original Message-----
>>> From: BJ Freeman [mailto:[hidden email]]
>>> Sent: November-09-10 6:00 PM
>>> To: [hidden email]
>>> Subject: Re: How to import a pre existing product database
>>>
>>> all web tools imports into memory so is limited to the amount of free memory of the sever.
>>>
>>> I imports files that are gigs in size so read in the files. I typical file has 25 columns and takes about 15min.
>>> There is no need to write code to convert files to xml since the
>>> https://cwiki.apache.org/confluence/display/OFBENDUSER/OFBiz%27s+Dat
>>> a+
>>> File+Tools
>>>
>>>
>>>
>>>
>>>
>>> =========================
>>> BJ Freeman
>>> Strategic Power Office with Supplier
>>> Automation<http://www.businessesnetwork.com/automation/viewforum.php?f
>>> =52> Specialtymarket.com<http://www.specialtymarket.com/>
>>> Systems Integrator-- Glad to Assist
>>>
>>> Chat Y! messenger: bjfr33man
>>> James McGill sent the following on 11/9/2010 12:38 PM:
>>>
>>>> On Tue, Nov 9, 2010 at 9:34 AM, James Lawton<[hidden email]>wrote:
>>>>
>>>>> How would I import a pre existing product database? I currently
>>>>> have it in a CSV and I am looking for a method of getting that
>>>>> information into the database so that the imported items will show
>>>>> up in the catalog
>>>>>
>>>>
>>>>
>>>> OFBiz ETL is best done as XML seed data.
>>>> In your scenario I would seriously consider converting the CSV into
>>>> XML (a job for Perl), and perhaps doing some input validation as
>>>> part of that conversion, and then loading the XML as seed data.
>>>>
>>>> If you really have a lot of ETL to do (e.g., processing that will
>>>> take hours or days to run through the service interface) you might
>>>> be better off going straight to the database. You lose some of the
>>>> traceability this way (e.g., logging and event side-effects) so it
>>>> isn't recommended. (We ran into problems when loading in several
>>>> years worth of historical data, orders and inventory transactions.)
>>>>
>>>> --
>>>> James McGill
>>>> Phoenix AZ
>>>>
>>>
>>>
>>>
>>
>>
>>
>
>
>

BJ Freeman

Re: How to import a pre existing product database

take a look at your logs.
please give detail steps

=========================
BJ Freeman
Strategic Power Office with Supplier Automation <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
Specialtymarket.com <http://www.specialtymarket.com/>
Systems Integrator-- Glad to Assist

Chat Y! messenger: bjfr33man

James Lawton sent the following on 12/1/2010 10:27 AM:

> Thanks for the link and the record information...
>
> I have altered the fields to fit the product record field names and tried to use the "work with data files" tool again
>
> I enter the Definition File Name in the screen
> "file://locahost/c:\files\tofino\import data\tofinodata.xml" and select the URL check box and then hit Run. The screen flickers but nothing happens. The instructions say that a drop down box will appear with a list of the definitions to pick from...
>
> Thoughts?
>
> Here is the modified definition file:
>
> <data-file name="c:\files\tofino\import data\itemdatabasereportmod.txt" separator-style="delimited" type-code="UTF-8"
> delimiter="&#009" text-delimiter=""
> description="this takes the product data from jeffs file"
> has-header="true">
> <record name="Product">
> <field name="manufacturerPartyId" ignored="false" type="String"/>
> <field name="productId" ignored="false" type="String"/>
> <field name="primaryProductCategoryId" expression="true"
> default-value="tools" type="String"/>
> <field name="productTypeId" expression="true"
> default-value="FINISHED_GOOD" type="String"/>
> <field name="productName" ignored="false" type="String"/>
> <field name="description" ignored="false" type="String"/>
> <field name="quantityIncluded" expression="true" default-value="1"
> type="String"/>
> <field name="priceDetailText" ignored="true" type="String"/>
> </record>
> </data-file>
>
>
> Thanks
> James
> -----Original Message-----
> From: BJ Freeman [mailto:[hidden email]]
> Sent: December-01-10 12:37 PM
> To: [hidden email]
> Subject: Re: How to import a pre existing product database
>
> if you look at the table the manufacture field is in then look up the same in the entities https://demo-trunk.ofbiz.apache.org/webtools/control/entitymaint
> Just to clarify
> your file has a Record=Product.
> this means you will be creating a Product entity file, so only fields in the product entity can be created.
> https://demo-trunk.ofbiz.apache.org/webtools/control/FindGeneric?entityName=Product&find=true&VIEW_SIZE=50&VIEW_INDEX=0
> if you have data that goes in different entities then you need to create a Record for each entity with its fields.
>
>
> James Lawton sent the following on 12/1/2010 9:16 AM:
>
>
> =========================
> BJ Freeman
> Strategic Power Office with Supplier Automation<http://www.businessesnetwork.com/automation/viewforum.php?f=52>
> Specialtymarket.com<http://www.specialtymarket.com/> Systems Integrator-- Glad to Assist
>
> Chat Y! messenger: bjfr33man
>
>> Thanks
>>
>> I must be missing the spot in web tools it has a complete list of the field names - could you point that out again? For example what is the right field name for the manufacturer / supplier? I did take that out the db....
>>
>> Thanks
>> James
>>
>> -----Original Message-----
>> From: BJ Freeman [mailto:[hidden email]]
>> Sent: November-30-10 11:42 PM
>> To: [hidden email]
>> Subject: Re: How to import a pre existing product database
>>
>> there should be more to the log that would pinpoint what is going on.
>> I am guessing it is your input to the form, since this has not been changed 09.
>> if you look at the file mentioned you will see
>> <script
>> location="component://webtools/webapp/webtools/WEB-INF/actions/datafil
>> e/viewdatafile.groovy"/>
>> that is where the error referenced.
>>
>> your xml(not xsd) should have field names as you see them in webtools.
>> when you have underscores like manufacturer_party_id then that is what is in the db but ofbiz will not recognize it in Webtools or code.
>>
>>
>>
>>
>> James Lawton sent the following on 11/30/2010 7:34 PM:
>>
>>
>>> I think I am getting closer - thanks for the tips and the patience!
>>>
>>> I have been through all the files and here is what I have gotten when
>>> running my definition file through the data tools
>>>
>>> org.ofbiz.widget.screen.ScreenRenderException: Error rendering screen
>>> [component://webtools/widget/MiscScreens.xml#viewdatafile]:
>>> groovy.lang.MissingPropertyException: No such property: definitionUrl
>>> for class: viewdatafile (No such property: definitionUrl for class:
>>> viewdatafile)
>>>
>>>
>>> Here is the xsd file I was running:
>>>
>>>
>>> <data-file name="itemdatabasereport.txt" separator-style="delimited" type-code="UTF-8"
>>> delimiter="&#009" text-delimiter=""
>>> description="this takes the product data from jeffs file"
>>> has-header="true">
>>> <record name="Product">
>>> <field name="manufacturer_party_id" ignored="false" type="String"/>
>>> <field name="productId" ignored="false" type="String"/>
>>> <field name="primaryProductCategoryId" expression="true"
>>> default-value="tools" type="String"/>
>>> <field name="productTypeId" expression="true"
>>> default-value="FINISHED_GOOD" type="String"/>
>>> <field name="productName" ignored="false" type="String"/>
>>> <field name="description" ignored="false" type="String"/>
>>> <field name="quantityIncluded" expression="true" default-value="1"
>>> type="String"/>
>>> <field name="price" ignored="true" type="String"/>
>>> </record>
>>> </data-file>
>>>
>>>
>>> What's wrong with my definition file which is causing the data tool to throw up?
>>>
>>> Thanks
>>> James
>>>
>>>
>>>
>>> -----Original Message-----
>>> From: BJ Freeman [mailto:[hidden email]]
>>> Sent: November-26-10 1:36 AM
>>> To: [hidden email]
>>> Subject: Re: How to import a pre existing product database
>>>
>>> I am not clear what you are doing.
>>> the example you show is for an entity DataImportProduct which does
>>> not exist in the OOTB ofbiz take a look at
>>>
>>> https://issues.apache.org/jira/browse/OFBIZ-2534
>>> for tab examples.
>>> also
>>> https://issues.apache.org/jira/browse/OFBIZ-2518
>>>
>>> =========================
>>>
>>> BJ Freeman
>>> Strategic Power Office with Supplier Automation<http://www.businessesnetwork.com/automation/viewforum.php?f=52>
>>> Specialtymarket.com<http://www.specialtymarket.com/> Systems Integrator-- Glad to Assist
>>>
>>> Chat Y! messenger: bjfr33man
>>> James Lawton sent the following on 11/25/2010 6:36 PM:
>>>> I am probably doing something stupid here...I have dug into the link
>>>> that was posted and it looks simple enough...however the code
>>>> snippet is for a fixed width file rather than a comma delimited
>>>> file...and my attempts to create the right schema for importing
>>>> product are failing...If someone has a CSV file snippet to share
>>>> that would be awesome
>>>>
>>>> So heading in a different direction I created the entity data snippet below:
>>>>
>>>>  <entity-engine-xml>
>>>>
>>>> <DataImportProduct productId="YG0081ATF" productTypeId="FINISHED_GOOD" description="1/8 x 1-59/64 CARBIDE DREAM DRILL W/O COOLANT HOLES (3XD)" price="22.13" priceCurrencyUomId="USD" supplierPartyId="YG1" purchasePrice=""/>
>>>> <DataImportProduct productId="YG0081BTF"
>>>> productTypeId="FINISHED_GOOD" description="1/8 x 15/64 CARBIDE DREAM
>>>> DRILL W/ COOLANT HOLE" price="105.73" priceCurrencyUomId="USD"
>>>> supplierPartyId="YG1" purchasePrice=""/>
>>>>
>>>>
>>>> </entity-engine-xml>
>>>>
>>>> However every time I have tried to use the XML IMport tool it just
>>>> keeps coming back and saying that the either URL / file or the XML
>>>> are not complete
>>>>
>>>> What am I doing wrong?
>>>>
>>>> -----Original Message-----
>>>> From: BJ Freeman [mailto:[hidden email]]
>>>> Sent: November-09-10 6:00 PM
>>>> To: [hidden email]
>>>> Subject: Re: How to import a pre existing product database
>>>>
>>>> all web tools imports into memory so is limited to the amount of free memory of the sever.
>>>>
>>>> I imports files that are gigs in size so read in the files. I typical file has 25 columns and takes about 15min.
>>>> There is no need to write code to convert files to xml since the
>>>> https://cwiki.apache.org/confluence/display/OFBENDUSER/OFBiz%27s+Dat
>>>> a+
>>>> File+Tools
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> =========================
>>>> BJ Freeman
>>>> Strategic Power Office with Supplier
>>>> Automation<http://www.businessesnetwork.com/automation/viewforum.php?f
>>>> =52> Specialtymarket.com<http://www.specialtymarket.com/>
>>>> Systems Integrator-- Glad to Assist
>>>>
>>>> Chat Y! messenger: bjfr33man
>>>> James McGill sent the following on 11/9/2010 12:38 PM:
>>>>
>>>>> On Tue, Nov 9, 2010 at 9:34 AM, James Lawton<[hidden email]>wrote:
>>>>>
>>>>>> How would I import a pre existing product database? I currently
>>>>>> have it in a CSV and I am looking for a method of getting that
>>>>>> information into the database so that the imported items will show
>>>>>> up in the catalog
>>>>>>
>>>>>
>>>>>
>>>>> OFBiz ETL is best done as XML seed data.
>>>>> In your scenario I would seriously consider converting the CSV into
>>>>> XML (a job for Perl), and perhaps doing some input validation as
>>>>> part of that conversion, and then loading the XML as seed data.
>>>>>
>>>>> If you really have a lot of ETL to do (e.g., processing that will
>>>>> take hours or days to run through the service interface) you might
>>>>> be better off going straight to the database. You lose some of the
>>>>> traceability this way (e.g., logging and event side-effects) so it
>>>>> isn't recommended. (We ran into problems when loading in several
>>>>> years worth of historical data, orders and inventory transactions.)
>>>>>
>>>>> --
>>>>> James McGill
>>>>> Phoenix AZ
>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>>
>
>
>

James Lawton

RE: How to import a pre existing product database

Here is the section of ofbiz.log for the timeframe

Here is what I did:

entered the file information and pressed "Run"

Thanks

James

-----Original Message-----
From: BJ Freeman [mailto:[hidden email]]
Sent: December-01-10 2:18 PM
To: [hidden email]
Subject: Re: How to import a pre existing product database

take a look at your logs.

please give detail steps

=========================

BJ Freeman

Strategic Power Office with Supplier Automation <http://www.businessesnetwork.com/automation/viewforum.php?f=52>

Specialtymarket.com <http://www.specialtymarket.com/> Systems Integrator-- Glad to Assist

Chat Y! messenger: bjfr33man

James Lawton sent the following on 12/1/2010 10:27 AM:

> Thanks for the link and the record information...

> I have altered the fields to fit the product record field names and

> tried to use the "work with data files" tool again

> I enter the Definition File Name in the screen

> "<a href="file:///\\locahost\c:\files\tofino\import%20data\tofinodata.xml">file://locahost/c:\files\tofino\import data\tofinodata.xml" and select the URL check box and then hit Run. The screen flickers but nothing happens. The instructions say that a drop down box will appear with a list of the definitions to pick from...

> Thoughts?

> Here is the modified definition file:

> <data-file name="c:\files\tofino\import data\itemdatabasereportmod.txt" separator-style="delimited" type-code="UTF-8"

> delimiter="&#009" text-delimiter=""

> description="this takes the product data from jeffs file"

> has-header="true">

> <record name="Product">

> <field name="manufacturerPartyId" ignored="false" type="String"/>

> <field name="productId" ignored="false" type="String"/>

> <field name="primaryProductCategoryId" expression="true"

> default-value="tools" type="String"/>

> <field name="productTypeId" expression="true"

> default-value="FINISHED_GOOD" type="String"/>

> <field name="productName" ignored="false" type="String"/>

> <field name="description" ignored="false" type="String"/>

> <field name="quantityIncluded" expression="true" default-value="1"

> type="String"/>

> <field name="priceDetailText" ignored="true" type="String"/>

> </record>

> </data-file>

> Thanks

> James

> -----Original Message-----

> From: BJ Freeman [mailto:[hidden email]]

> Sent: December-01-10 12:37 PM

> To: [hidden email]

> Subject: Re: How to import a pre existing product database

> if you look at the table the manufacture field is in then look up the

> same in the entities

> https://demo-trunk.ofbiz.apache.org/webtools/control/entitymaint

> Just to clarify

> your file has a Record=Product.

> this means you will be creating a Product entity file, so only fields in the product entity can be created.

> https://demo-trunk.ofbiz.apache.org/webtools/control/FindGeneric?entit

> yName=Product&find=true&VIEW_SIZE=50&VIEW_INDEX=0

> if you have data that goes in different entities then you need to create a Record for each entity with its fields.

> James Lawton sent the following on 12/1/2010 9:16 AM:

> =========================

> BJ Freeman

> Strategic Power Office with Supplier

> Automation<http://www.businessesnetwork.com/automation/viewforum.php?f

> =52> Specialtymarket.com<http://www.specialtymarket.com/> Systems

> Integrator-- Glad to Assist

> Chat Y! messenger: bjfr33man

>> Thanks

>> I must be missing the spot in web tools it has a complete list of the field names - could you point that out again? For example what is the right field name for the manufacturer / supplier? I did take that out the db....

>> Thanks

>> James

>> -----Original Message-----

>> From: BJ Freeman [mailto:[hidden email]]

>> Sent: November-30-10 11:42 PM

>> To: [hidden email]

>> Subject: Re: How to import a pre existing product database

>> there should be more to the log that would pinpoint what is going on.

>> I am guessing it is your input to the form, since this has not been changed 09.

>> if you look at the file mentioned you will see

>> <script

>> location="component://webtools/webapp/webtools/WEB-INF/actions/datafi

>> l

>> e/viewdatafile.groovy"/>

>> that is where the error referenced.

>> your xml(not xsd) should have field names as you see them in webtools.

>> when you have underscores like manufacturer_party_id then that is what is in the db but ofbiz will not recognize it in Webtools or code.

>> James Lawton sent the following on 11/30/2010 7:34 PM:

>>> I think I am getting closer - thanks for the tips and the patience!

>>>

>>> I have been through all the files and here is what I have gotten

>>> when running my definition file through the data tools

>>>

>>> org.ofbiz.widget.screen.ScreenRenderException: Error rendering

>>> screen

>>> [component://webtools/widget/MiscScreens.xml#viewdatafile]:

>>> groovy.lang.MissingPropertyException: No such property:

>>> definitionUrl for class: viewdatafile (No such property: definitionUrl for class:

>>> viewdatafile)

>>>

>>> Here is the xsd file I was running:

>>>

>>> <data-file name="itemdatabasereport.txt" separator-style="delimited" type-code="UTF-8"

>>> delimiter="&#009" text-delimiter=""

>>> description="this takes the product data from jeffs file"

>>> has-header="true">

>>> <record name="Product">

>>> <field name="manufacturer_party_id" ignored="false" type="String"/>

>>> <field name="productId" ignored="false" type="String"/>

>>> <field name="primaryProductCategoryId" expression="true"

>>> default-value="tools" type="String"/>

>>> <field name="productTypeId" expression="true"

>>> default-value="FINISHED_GOOD" type="String"/>

>>> <field name="productName" ignored="false" type="String"/>

>>> <field name="description" ignored="false" type="String"/>

>>> <field name="quantityIncluded" expression="true" default-value="1"

>>> type="String"/>

>>> <field name="price" ignored="true" type="String"/>

>>> </record>

>>> </data-file>

>>>

>>> What's wrong with my definition file which is causing the data tool to throw up?

>>>

>>> Thanks

>>> James

>>>

>>> -----Original Message-----

>>> From: BJ Freeman [mailto:[hidden email]]

>>> Sent: November-26-10 1:36 AM

>>> To: [hidden email]

>>> Subject: Re: How to import a pre existing product database

>>>

>>> I am not clear what you are doing.

>>> the example you show is for an entity DataImportProduct which does

>>> not exist in the OOTB ofbiz take a look at

>>>

>>> https://issues.apache.org/jira/browse/OFBIZ-2534

>>> for tab examples.

>>> also

>>> https://issues.apache.org/jira/browse/OFBIZ-2518

>>>

>>> =========================

>>>

>>> BJ Freeman

>>> Strategic Power Office with Supplier Automation<http://www.businessesnetwork.com/automation/viewforum.php?f=52>

>>> Specialtymarket.com<http://www.specialtymarket.com/> Systems Integrator-- Glad to Assist

>>>

>>> Chat Y! messenger: bjfr33man

>>> James Lawton sent the following on 11/25/2010 6:36 PM:

>>>> I am probably doing something stupid here...I have dug into the

>>>> link that was posted and it looks simple enough...however the code

>>>> snippet is for a fixed width file rather than a comma delimited

>>>> file...and my attempts to create the right schema for importing

>>>> product are failing...If someone has a CSV file snippet to share

>>>> that would be awesome

>>>>

>>>> So heading in a different direction I created the entity data snippet below:

>>>>

>>>> <!-- this file contains some sample products for testing the data

>>>> import routine --> <entity-engine-xml>

>>>>

>>>> <DataImportProduct productId="YG0081ATF" productTypeId="FINISHED_GOOD" description="1/8 x 1-59/64 CARBIDE DREAM DRILL W/O COOLANT HOLES (3XD)" price="22.13" priceCurrencyUomId="USD" supplierPartyId="YG1" purchasePrice=""/>

>>>> <DataImportProduct productId="YG0081BTF"

>>>> productTypeId="FINISHED_GOOD" description="1/8 x 15/64 CARBIDE

>>>> DREAM DRILL W/ COOLANT HOLE" price="105.73" priceCurrencyUomId="USD"

>>>> supplierPartyId="YG1" purchasePrice=""/>

>>>>

>>>> </entity-engine-xml>

>>>>

>>>> However every time I have tried to use the XML IMport tool it just

>>>> keeps coming back and saying that the either URL / file or the XML

>>>> are not complete

>>>>

>>>> What am I doing wrong?

>>>>

>>>> -----Original Message-----

>>>> From: BJ Freeman [mailto:[hidden email]]

>>>> Sent: November-09-10 6:00 PM

>>>> To: [hidden email]

>>>> Subject: Re: How to import a pre existing product database

>>>>

>>>> all web tools imports into memory so is limited to the amount of free memory of the sever.

>>>>

>>>> I imports files that are gigs in size so read in the files. I typical file has 25 columns and takes about 15min.

>>>> There is no need to write code to convert files to xml since the

>>>> https://cwiki.apache.org/confluence/display/OFBENDUSER/OFBiz%27s+Da

>>>> t

>>>> a+

>>>> File+Tools

>>>>

>>>> =========================

>>>> BJ Freeman

>>>> Strategic Power Office with Supplier

>>>> Automation<http://www.businessesnetwork.com/automation/viewforum.php?f

>>>> =52> Specialtymarket.com<http://www.specialtymarket.com/>

>>>> Systems Integrator-- Glad to Assist

>>>>

>>>> Chat Y! messenger: bjfr33man

>>>> James McGill sent the following on 11/9/2010 12:38 PM:

>>>>

>>>>> On Tue, Nov 9, 2010 at 9:34 AM, James Lawton<[hidden email]>wrote:

>>>>>

>>>>>> How would I import a pre existing product database? I currently

>>>>>> have it in a CSV and I am looking for a method of getting that

>>>>>> information into the database so that the imported items will

>>>>>> show up in the catalog

>>>>>>

>>>>>

>>>>> OFBiz ETL is best done as XML seed data.

>>>>> In your scenario I would seriously consider converting the CSV

>>>>> into XML (a job for Perl), and perhaps doing some input validation

>>>>> as part of that conversion, and then loading the XML as seed data.

>>>>>

>>>>> If you really have a lot of ETL to do (e.g., processing that will

>>>>> take hours or days to run through the service interface) you might

>>>>> be better off going straight to the database. You lose some of

>>>>> the traceability this way (e.g., logging and event side-effects)

>>>>> so it isn't recommended. (We ran into problems when loading in

>>>>> several years worth of historical data, orders and inventory

>>>>> transactions.)

>>>>>

>>>>> --

>>>>> James McGill

>>>>> Phoenix AZ

>>>>>

>>>>

>>>

BJ Freeman

Re: How to import a pre existing product database

sorry see no logs and not enough detail.
I have no way to replicated your steps.

=========================
BJ Freeman
Strategic Power Office with Supplier Automation <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
Specialtymarket.com <http://www.specialtymarket.com/>
Systems Integrator-- Glad to Assist

Chat Y! messenger: bjfr33man

James Lawton sent the following on 12/1/2010 12:23 PM:

> Here is the section of ofbiz.log for the timeframe
>
>
>
>
>
>
>
>
>
> Here is what I did:
>
> entered the file information and pressed "Run"
>
>
>
>
>
>
>
>
>
> Thanks
>
> James
>
>
>
>
>
> -----Original Message-----
> From: BJ Freeman [mailto:[hidden email]]
> Sent: December-01-10 2:18 PM
> To: [hidden email]
> Subject: Re: How to import a pre existing product database
>
>
>
> take a look at your logs.
>
> please give detail steps
>
>
>
>
>
> =========================
>
> BJ Freeman
>
> Strategic Power Office with Supplier Automation< <http://www.businessesnetwork.com/automation/viewforum.php?f=52> http://www.businessesnetwork.com/automation/viewforum.php?f=52>
>
> Specialtymarket.com< <http://www.specialtymarket.com/> http://www.specialtymarket.com/> Systems Integrator-- Glad to Assist
>
>
>
> Chat Y! messenger: bjfr33man
>
>
>
>
>
> James Lawton sent the following on 12/1/2010 10:27 AM:
>
>> Thanks for the link and the record information...
>
>>
>
>> I have altered the fields to fit the product record field names and
>
>> tried to use the "work with data files" tool again
>
>>
>
>> I enter the Definition File Name in the screen
>
>> "<file:///\\locahost\c:\files\tofino\import%20data\tofinodata.xml> file://locahost/c:\files\tofino\import data\tofinodata.xml" and select the URL check box and then hit Run. The screen flickers but nothing happens. The instructions say that a drop down box will appear with a list of the definitions to pick from...
>
>>
>
>> Thoughts?
>
>>
>
>> Here is the modified definition file:
>
>>
>
>> <data-file name="c:\files\tofino\import data\itemdatabasereportmod.txt" separator-style="delimited" type-code="UTF-8"
>
>> delimiter="&#009" text-delimiter=""
>
>> description="this takes the product data from jeffs file"
>
>> has-header="true">
>
>> <record name="Product">
>
>> <field name="manufacturerPartyId" ignored="false" type="String"/>
>
>> <field name="productId" ignored="false" type="String"/>
>
>> <field name="primaryProductCategoryId" expression="true"
>
>> default-value="tools" type="String"/>
>
>> <field name="productTypeId" expression="true"
>
>> default-value="FINISHED_GOOD" type="String"/>
>
>> <field name="productName" ignored="false" type="String"/>
>
>> <field name="description" ignored="false" type="String"/>
>
>> <field name="quantityIncluded" expression="true" default-value="1"
>
>> type="String"/>
>
>> <field name="priceDetailText" ignored="true" type="String"/>
>
>> </record>
>
>> </data-file>
>
>>
>
>>
>
>> Thanks
>
>> James
>
>> -----Original Message-----
>
>> From: BJ Freeman [mailto:[hidden email]]
>
>> Sent: December-01-10 12:37 PM
>
>> To: [hidden email]
>
>> Subject: Re: How to import a pre existing product database
>
>>
>
>> if you look at the table the manufacture field is in then look up the
>
>> same in the entities
>
>> <https://demo-trunk.ofbiz.apache.org/webtools/control/entitymaint> https://demo-trunk.ofbiz.apache.org/webtools/control/entitymaint
>
>> Just to clarify
>
>> your file has a Record=Product.
>
>> this means you will be creating a Product entity file, so only fields in the product entity can be created.
>
>> <https://demo-trunk.ofbiz.apache.org/webtools/control/FindGeneric?entit> https://demo-trunk.ofbiz.apache.org/webtools/control/FindGeneric?entit
>
>> yName=Product&find=true&VIEW_SIZE=50&VIEW_INDEX=0
>
>> if you have data that goes in different entities then you need to create a Record for each entity with its fields.
>
>>
>
>>
>
>> James Lawton sent the following on 12/1/2010 9:16 AM:
>
>>
>
>>
>
>> =========================
>
>> BJ Freeman
>
>> Strategic Power Office with Supplier
>
>> Automation<http://www.businessesnetwork.com/automation/viewforum.php?f
>
>> =52> Specialtymarket.com< <http://www.specialtymarket.com/> http://www.specialtymarket.com/> Systems
>
>> Integrator-- Glad to Assist
>
>>
>
>> Chat Y! messenger: bjfr33man
>
>>
>
>>> Thanks
>
>>>
>
>>> I must be missing the spot in web tools it has a complete list of the field names - could you point that out again? For example what is the right field name for the manufacturer / supplier? I did take that out the db....
>
>>>
>
>>> Thanks
>
>>> James
>
>>>
>
>>> -----Original Message-----
>
>>> From: BJ Freeman [mailto:[hidden email]]
>
>>> Sent: November-30-10 11:42 PM
>
>>> To: [hidden email]
>
>>> Subject: Re: How to import a pre existing product database
>
>>>
>
>>> there should be more to the log that would pinpoint what is going on.
>
>>> I am guessing it is your input to the form, since this has not been changed 09.
>
>>> if you look at the file mentioned you will see
>
>>> <script
>
>>> location="component://webtools/webapp/webtools/WEB-INF/actions/datafi
>
>>> l
>
>>> e/viewdatafile.groovy"/>
>
>>> that is where the error referenced.
>
>>>
>
>>> your xml(not xsd) should have field names as you see them in webtools.
>
>>> when you have underscores like manufacturer_party_id then that is what is in the db but ofbiz will not recognize it in Webtools or code.
>
>>>
>
>>>
>
>>>
>
>>>
>
>>> James Lawton sent the following on 11/30/2010 7:34 PM:
>
>>>
>
>>>
>
>>>> I think I am getting closer - thanks for the tips and the patience!
>
>>>>
>
>>>> I have been through all the files and here is what I have gotten
>
>>>> when running my definition file through the data tools
>
>>>>
>
>>>> org.ofbiz.widget.screen.ScreenRenderException: Error rendering
>
>>>> screen
>
>>>> [component://webtools/widget/MiscScreens.xml#viewdatafile]:
>
>>>> groovy.lang.MissingPropertyException: No such property:
>
>>>> definitionUrl for class: viewdatafile (No such property: definitionUrl for class:
>
>>>> viewdatafile)
>
>>>>
>
>>>>
>
>>>> Here is the xsd file I was running:
>
>>>>
>
>>>>
>
>>>> <data-file name="itemdatabasereport.txt" separator-style="delimited" type-code="UTF-8"
>
>>>> delimiter="&#009" text-delimiter=""
>
>>>> description="this takes the product data from jeffs file"
>
>>>> has-header="true">
>
>>>> <record name="Product">
>
>>>> <field name="manufacturer_party_id" ignored="false" type="String"/>
>
>>>> <field name="productId" ignored="false" type="String"/>
>
>>>> <field name="primaryProductCategoryId" expression="true"
>
>>>> default-value="tools" type="String"/>
>
>>>> <field name="productTypeId" expression="true"
>
>>>> default-value="FINISHED_GOOD" type="String"/>
>
>>>> <field name="productName" ignored="false" type="String"/>
>
>>>> <field name="description" ignored="false" type="String"/>
>
>>>> <field name="quantityIncluded" expression="true" default-value="1"
>
>>>> type="String"/>
>
>>>> <field name="price" ignored="true" type="String"/>
>
>>>> </record>
>
>>>> </data-file>
>
>>>>
>
>>>>
>
>>>> What's wrong with my definition file which is causing the data tool to throw up?
>
>>>>
>
>>>> Thanks
>
>>>> James
>
>>>>
>
>>>>
>
>>>>
>
>>>> -----Original Message-----
>
>>>> From: BJ Freeman [mailto:[hidden email]]
>
>>>> Sent: November-26-10 1:36 AM
>
>>>> To: [hidden email]
>
>>>> Subject: Re: How to import a pre existing product database
>
>>>>
>
>>>> I am not clear what you are doing.
>
>>>> the example you show is for an entity DataImportProduct which does
>
>>>> not exist in the OOTB ofbiz take a look at
>
>>>>
>
>>>> <https://issues.apache.org/jira/browse/OFBIZ-2534> https://issues.apache.org/jira/browse/OFBIZ-2534
>
>>>> for tab examples.
>
>>>> also
>
>>>> <https://issues.apache.org/jira/browse/OFBIZ-2518> https://issues.apache.org/jira/browse/OFBIZ-2518
>
>>>>
>
>>>> =========================
>
>>>>
>
>>>> BJ Freeman
>
>>>> Strategic Power Office with Supplier Automation< <http://www.businessesnetwork.com/automation/viewforum.php?f=52> http://www.businessesnetwork.com/automation/viewforum.php?f=52>
>
>>>> Specialtymarket.com< <http://www.specialtymarket.com/> http://www.specialtymarket.com/> Systems Integrator-- Glad to Assist
>
>>>>
>
>>>> Chat Y! messenger: bjfr33man
>
>>>> James Lawton sent the following on 11/25/2010 6:36 PM:
>
>>>>> I am probably doing something stupid here...I have dug into the
>
>>>>> link that was posted and it looks simple enough...however the code
>
>>>>> snippet is for a fixed width file rather than a comma delimited
>
>>>>> file...and my attempts to create the right schema for importing
>
>>>>> product are failing...If someone has a CSV file snippet to share
>
>>>>> that would be awesome
>
>>>>>
>
>>>>> So heading in a different direction I created the entity data snippet below:
>
>>>>>
>
>>>>>  <entity-engine-xml>
>
>>>>>
>
>>>>> <DataImportProduct productId="YG0081ATF" productTypeId="FINISHED_GOOD" description="1/8 x 1-59/64 CARBIDE DREAM DRILL W/O COOLANT HOLES (3XD)" price="22.13" priceCurrencyUomId="USD" supplierPartyId="YG1" purchasePrice=""/>
>
>>>>> <DataImportProduct productId="YG0081BTF"
>
>>>>> productTypeId="FINISHED_GOOD" description="1/8 x 15/64 CARBIDE
>
>>>>> DREAM DRILL W/ COOLANT HOLE" price="105.73" priceCurrencyUomId="USD"
>
>>>>> supplierPartyId="YG1" purchasePrice=""/>
>
>>>>>
>
>>>>>
>
>>>>> </entity-engine-xml>
>
>>>>>
>
>>>>> However every time I have tried to use the XML IMport tool it just
>
>>>>> keeps coming back and saying that the either URL / file or the XML
>
>>>>> are not complete
>
>>>>>
>
>>>>> What am I doing wrong?
>
>>>>>
>
>>>>> -----Original Message-----
>
>>>>> From: BJ Freeman [mailto:[hidden email]]
>
>>>>> Sent: November-09-10 6:00 PM
>
>>>>> To: [hidden email]
>
>>>>> Subject: Re: How to import a pre existing product database
>
>>>>>
>
>>>>> all web tools imports into memory so is limited to the amount of free memory of the sever.
>
>>>>>
>
>>>>> I imports files that are gigs in size so read in the files. I typical file has 25 columns and takes about 15min.
>
>>>>> There is no need to write code to convert files to xml since the
>
>>>>> <https://cwiki.apache.org/confluence/display/OFBENDUSER/OFBiz%27s+Da> https://cwiki.apache.org/confluence/display/OFBENDUSER/OFBiz%27s+Da
>
>>>>> t
>
>>>>> a+
>
>>>>> File+Tools
>
>>>>>
>
>>>>>
>
>>>>>
>
>>>>>
>
>>>>>
>
>>>>> =========================
>
>>>>> BJ Freeman
>
>>>>> Strategic Power Office with Supplier
>
>>>>> Automation<http://www.businessesnetwork.com/automation/viewforum.php?f
>
>>>>> =52> Specialtymarket.com< <http://www.specialtymarket.com/> http://www.specialtymarket.com/>
>
>>>>> Systems Integrator-- Glad to Assist
>
>>>>>
>
>>>>> Chat Y! messenger: bjfr33man
>
>>>>> James McGill sent the following on 11/9/2010 12:38 PM:
>
>>>>>
>
>>>>>> On Tue, Nov 9, 2010 at 9:34 AM, James Lawton< <mailto:[hidden email]> [hidden email]>wrote:
>
>>>>>>
>
>>>>>>> How would I import a pre existing product database? I currently
>
>>>>>>> have it in a CSV and I am looking for a method of getting that
>
>>>>>>> information into the database so that the imported items will
>
>>>>>>> show up in the catalog
>
>>>>>>>
>
>>>>>>
>
>>>>>>
>
>>>>>> OFBiz ETL is best done as XML seed data.
>
>>>>>> In your scenario I would seriously consider converting the CSV
>
>>>>>> into XML (a job for Perl), and perhaps doing some input validation
>
>>>>>> as part of that conversion, and then loading the XML as seed data.
>
>>>>>>
>
>>>>>> If you really have a lot of ETL to do (e.g., processing that will
>
>>>>>> take hours or days to run through the service interface) you might
>
>>>>>> be better off going straight to the database. You lose some of
>
>>>>>> the traceability this way (e.g., logging and event side-effects)
>
>>>>>> so it isn't recommended. (We ran into problems when loading in
>
>>>>>> several years worth of historical data, orders and inventory
>
>>>>>> transactions.)
>
>>>>>>
>
>>>>>> --
>
>>>>>> James McGill
>
>>>>>> Phoenix AZ
>
>>>>>>
>
>>>>>
>
>>>>>
>
>>>>>
>
>>>>
>
>>>>
>
>>>>
>
>>>
>
>>>
>
>>>
>
>>
>
>>
>
>>
>
>

James Lawton

RE: [SPAM]Re: How to import a pre existing product database

Hmmm, not sure what happened there - I had appended the logs and a snapshot of the working with files screen...

Is there something which strips out attachments?

I'll try again

-----Original Message-----
From: BJ Freeman [mailto:[hidden email]]
Sent: December-01-10 3:41 PM
To: [hidden email]
Subject: [SPAM]Re: How to import a pre existing product database

sorry see no logs and not enough detail.
I have no way to replicated your steps.

=========================
BJ Freeman
Strategic Power Office with Supplier Automation <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
Specialtymarket.com <http://www.specialtymarket.com/> Systems Integrator-- Glad to Assist

Chat Y! messenger: bjfr33man

James Lawton sent the following on 12/1/2010 12:23 PM:

> Here is the section of ofbiz.log for the timeframe
>
>
>
>
>
>
>
>
>
> Here is what I did:
>
> entered the file information and pressed "Run"
>
>
>
>
>
>
>
>
>
> Thanks
>
> James
>
>
>
>
>
> -----Original Message-----
> From: BJ Freeman [mailto:[hidden email]]
> Sent: December-01-10 2:18 PM
> To: [hidden email]
> Subject: [SPAM]Re: How to import a pre existing product database
>
>
>
> take a look at your logs.
>
> please give detail steps
>
>
>
>
>
> =========================
>
> BJ Freeman
>
> Strategic Power Office with Supplier Automation<
> <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
> http://www.businessesnetwork.com/automation/viewforum.php?f=52>
>
> Specialtymarket.com< <http://www.specialtymarket.com/>
> http://www.specialtymarket.com/> Systems Integrator-- Glad to Assist
>
>
>
> Chat Y! messenger: bjfr33man
>
>
>
>
>
> James Lawton sent the following on 12/1/2010 10:27 AM:
>
>> Thanks for the link and the record information...
>
>>
>
>> I have altered the fields to fit the product record field names and
>
>> tried to use the "work with data files" tool again
>
>>
>
>> I enter the Definition File Name in the screen
>
>> "<file:///\\locahost\c:\files\tofino\import%20data\tofinodata.xml> file://locahost/c:\files\tofino\import data\tofinodata.xml" and select the URL check box and then hit Run. The screen flickers but nothing happens. The instructions say that a drop down box will appear with a list of the definitions to pick from...
>
>>
>
>> Thoughts?
>
>>
>
>> Here is the modified definition file:
>
>>
>
>> <data-file name="c:\files\tofino\import data\itemdatabasereportmod.txt" separator-style="delimited" type-code="UTF-8"
>
>> delimiter="&#009" text-delimiter=""
>
>> description="this takes the product data from jeffs file"
>
>> has-header="true">
>
>> <record name="Product">
>
>> <field name="manufacturerPartyId" ignored="false"
>> type="String"/>
>
>> <field name="productId" ignored="false" type="String"/>
>
>> <field name="primaryProductCategoryId" expression="true"
>
>> default-value="tools" type="String"/>
>
>> <field name="productTypeId" expression="true"
>
>> default-value="FINISHED_GOOD" type="String"/>
>
>> <field name="productName" ignored="false" type="String"/>
>
>> <field name="description" ignored="false" type="String"/>
>
>> <field name="quantityIncluded" expression="true" default-value="1"
>
>> type="String"/>
>
>> <field name="priceDetailText" ignored="true"
>> type="String"/>
>
>> </record>
>
>> </data-file>
>
>>
>
>>
>
>> Thanks
>
>> James
>
>> -----Original Message-----
>
>> From: BJ Freeman [mailto:[hidden email]]
>
>> Sent: December-01-10 12:37 PM
>
>> To: [hidden email]
>
>> Subject: [SPAM]Re: How to import a pre existing product database
>
>>
>
>> if you look at the table the manufacture field is in then look up the
>
>> same in the entities
>
>> <https://demo-trunk.ofbiz.apache.org/webtools/control/entitymaint>
>> https://demo-trunk.ofbiz.apache.org/webtools/control/entitymaint
>
>> Just to clarify
>
>> your file has a Record=Product.
>
>> this means you will be creating a Product entity file, so only fields in the product entity can be created.
>
>>
>> <https://demo-trunk.ofbiz.apache.org/webtools/control/FindGeneric?ent
>> it>
>> https://demo-trunk.ofbiz.apache.org/webtools/control/FindGeneric?enti
>> t
>
>> yName=Product&find=true&VIEW_SIZE=50&VIEW_INDEX=0
>
>> if you have data that goes in different entities then you need to create a Record for each entity with its fields.
>
>>
>
>>
>
>> James Lawton sent the following on 12/1/2010 9:16 AM:
>
>>
>
>>
>
>> =========================
>
>> BJ Freeman
>
>> Strategic Power Office with Supplier
>
>> Automation<http://www.businessesnetwork.com/automation/viewforum.php?
>> f
>
>> =52> Specialtymarket.com< <http://www.specialtymarket.com/> http://www.specialtymarket.com/> Systems
>
>> Integrator-- Glad to Assist
>
>>
>
>> Chat Y! messenger: bjfr33man
>
>>
>
>>> Thanks
>
>>>
>
>>> I must be missing the spot in web tools it has a complete list of the field names - could you point that out again? For example what is the right field name for the manufacturer / supplier? I did take that out the db....
>
>>>
>
>>> Thanks
>
>>> James
>
>>>
>
>>> -----Original Message-----
>
>>> From: BJ Freeman [mailto:[hidden email]]
>
>>> Sent: November-30-10 11:42 PM
>
>>> To: [hidden email]
>
>>> Subject: [SPAM]Re: How to import a pre existing product database
>
>>>
>
>>> there should be more to the log that would pinpoint what is going on.
>
>>> I am guessing it is your input to the form, since this has not been changed 09.
>
>>> if you look at the file mentioned you will see
>
>>> <script
>
>>> location="component://webtools/webapp/webtools/WEB-INF/actions/dataf
>>> i
>
>>> l
>
>>> e/viewdatafile.groovy"/>
>
>>> that is where the error referenced.
>
>>>
>
>>> your xml(not xsd) should have field names as you see them in webtools.
>
>>> when you have underscores like manufacturer_party_id then that is what is in the db but ofbiz will not recognize it in Webtools or code.
>
>>>
>
>>>
>
>>>
>
>>>
>
>>> James Lawton sent the following on 11/30/2010 7:34 PM:
>
>>>
>
>>>
>
>>>> I think I am getting closer - thanks for the tips and the patience!
>
>>>>
>
>>>> I have been through all the files and here is what I have gotten
>
>>>> when running my definition file through the data tools
>
>>>>
>
>>>> org.ofbiz.widget.screen.ScreenRenderException: Error rendering
>
>>>> screen
>
>>>> [component://webtools/widget/MiscScreens.xml#viewdatafile]:
>
>>>> groovy.lang.MissingPropertyException: No such property:
>
>>>> definitionUrl for class: viewdatafile (No such property: definitionUrl for class:
>
>>>> viewdatafile)
>
>>>>
>
>>>>
>
>>>> Here is the xsd file I was running:
>
>>>>
>
>>>>
>
>>>> <data-file name="itemdatabasereport.txt" separator-style="delimited" type-code="UTF-8"
>
>>>> delimiter="&#009" text-delimiter=""
>
>>>> description="this takes the product data from jeffs file"
>
>>>> has-header="true">
>
>>>> <record name="Product">
>
>>>> <field name="manufacturer_party_id" ignored="false"
>>>> type="String"/>
>
>>>> <field name="productId" ignored="false" type="String"/>
>
>>>> <field name="primaryProductCategoryId" expression="true"
>
>>>> default-value="tools" type="String"/>
>
>>>> <field name="productTypeId" expression="true"
>
>>>> default-value="FINISHED_GOOD" type="String"/>
>
>>>> <field name="productName" ignored="false"
>>>> type="String"/>
>
>>>> <field name="description" ignored="false"
>>>> type="String"/>
>
>>>> <field name="quantityIncluded" expression="true" default-value="1"
>
>>>> type="String"/>
>
>>>> <field name="price" ignored="true" type="String"/>
>
>>>> </record>
>
>>>> </data-file>
>
>>>>
>
>>>>
>
>>>> What's wrong with my definition file which is causing the data tool to throw up?
>
>>>>
>
>>>> Thanks
>
>>>> James
>
>>>>
>
>>>>
>
>>>>
>
>>>> -----Original Message-----
>
>>>> From: BJ Freeman [mailto:[hidden email]]
>
>>>> Sent: November-26-10 1:36 AM
>
>>>> To: [hidden email]
>
>>>> Subject: [SPAM]Re: How to import a pre existing product database
>
>>>>
>
>>>> I am not clear what you are doing.
>
>>>> the example you show is for an entity DataImportProduct which does
>
>>>> not exist in the OOTB ofbiz take a look at
>
>>>>
>
>>>> <https://issues.apache.org/jira/browse/OFBIZ-2534>
>>>> https://issues.apache.org/jira/browse/OFBIZ-2534
>
>>>> for tab examples.
>
>>>> also
>
>>>> <https://issues.apache.org/jira/browse/OFBIZ-2518>
>>>> https://issues.apache.org/jira/browse/OFBIZ-2518
>
>>>>
>
>>>> =========================
>
>>>>
>
>>>> BJ Freeman
>
>>>> Strategic Power Office with Supplier Automation<
>>>> <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
>>>> http://www.businessesnetwork.com/automation/viewforum.php?f=52>
>
>>>> Specialtymarket.com< <http://www.specialtymarket.com/> http://www.specialtymarket.com/> Systems Integrator-- Glad to Assist
>
>>>>
>
>>>> Chat Y! messenger: bjfr33man
>
>>>> James Lawton sent the following on 11/25/2010 6:36 PM:
>
>>>>> I am probably doing something stupid here...I have dug into the
>
>>>>> link that was posted and it looks simple enough...however the code
>
>>>>> snippet is for a fixed width file rather than a comma delimited
>
>>>>> file...and my attempts to create the right schema for importing
>
>>>>> product are failing...If someone has a CSV file snippet to share
>
>>>>> that would be awesome
>
>>>>>
>
>>>>> So heading in a different direction I created the entity data snippet below:
>
>>>>>
>
>>>>>  <entity-engine-xml>
>
>>>>>
>
>>>>> <DataImportProduct productId="YG0081ATF"
>>>>> productTypeId="FINISHED_GOOD" description="1/8 x 1-59/64 CARBIDE
>>>>> DREAM DRILL W/O COOLANT HOLES (3XD)" price="22.13"
>>>>> priceCurrencyUomId="USD" supplierPartyId="YG1" purchasePrice=""/>
>
>>>>> <DataImportProduct productId="YG0081BTF"
>
>>>>> productTypeId="FINISHED_GOOD" description="1/8 x 15/64 CARBIDE
>
>>>>> DREAM DRILL W/ COOLANT HOLE" price="105.73" priceCurrencyUomId="USD"
>
>>>>> supplierPartyId="YG1" purchasePrice=""/>
>
>>>>>
>
>>>>>
>
>>>>> </entity-engine-xml>
>
>>>>>
>
>>>>> However every time I have tried to use the XML IMport tool it just
>
>>>>> keeps coming back and saying that the either URL / file or the XML
>
>>>>> are not complete
>
>>>>>
>
>>>>> What am I doing wrong?
>
>>>>>
>
>>>>> -----Original Message-----
>
>>>>> From: BJ Freeman [mailto:[hidden email]]
>
>>>>> Sent: November-09-10 6:00 PM
>
>>>>> To: [hidden email]
>
>>>>> Subject: [SPAM]Re: How to import a pre existing product database
>
>>>>>
>
>>>>> all web tools imports into memory so is limited to the amount of free memory of the sever.
>
>>>>>
>
>>>>> I imports files that are gigs in size so read in the files. I typical file has 25 columns and takes about 15min.
>
>>>>> There is no need to write code to convert files to xml since the
>
>>>>>
>>>>> <https://cwiki.apache.org/confluence/display/OFBENDUSER/OFBiz%27s+
>>>>> Da>
>>>>> https://cwiki.apache.org/confluence/display/OFBENDUSER/OFBiz%27s+D
>>>>> a
>
>>>>> t
>
>>>>> a+
>
>>>>> File+Tools
>
>>>>>
>
>>>>>
>
>>>>>
>
>>>>>
>
>>>>>
>
>>>>> =========================
>
>>>>> BJ Freeman
>
>>>>> Strategic Power Office with Supplier
>
>>>>> Automation<http://www.businessesnetwork.com/automation/viewforum.p
>>>>> hp?f
>
>>>>> =52> Specialtymarket.com< <http://www.specialtymarket.com/> http://www.specialtymarket.com/>
>
>>>>> Systems Integrator-- Glad to Assist
>
>>>>>
>
>>>>> Chat Y! messenger: bjfr33man
>
>>>>> James McGill sent the following on 11/9/2010 12:38 PM:
>
>>>>>
>
>>>>>> On Tue, Nov 9, 2010 at 9:34 AM, James Lawton< <mailto:[hidden email]> [hidden email]>wrote:
>
>>>>>>
>
>>>>>>> How would I import a pre existing product database? I currently
>
>>>>>>> have it in a CSV and I am looking for a method of getting that
>
>>>>>>> information into the database so that the imported items will
>
>>>>>>> show up in the catalog
>
>>>>>>>
>
>>>>>>
>
>>>>>>
>
>>>>>> OFBiz ETL is best done as XML seed data.
>
>>>>>> In your scenario I would seriously consider converting the CSV
>
>>>>>> into XML (a job for Perl), and perhaps doing some input
>>>>>> validation
>
>>>>>> as part of that conversion, and then loading the XML as seed data.
>
>>>>>>
>
>>>>>> If you really have a lot of ETL to do (e.g., processing that will
>
>>>>>> take hours or days to run through the service interface) you
>>>>>> might
>
>>>>>> be better off going straight to the database. You lose some of
>
>>>>>> the traceability this way (e.g., logging and event side-effects)
>
>>>>>> so it isn't recommended. (We ran into problems when loading in
>
>>>>>> several years worth of historical data, orders and inventory
>
>>>>>> transactions.)
>
>>>>>>
>
>>>>>> --
>
>>>>>> James McGill
>
>>>>>> Phoenix AZ
>
>>>>>>
>
>>>>>
>
>>>>>
>
>>>>>
>
>>>>
>
>>>>
>
>>>>
>
>>>
>
>>>
>
>>>
>
>>
>
>>
>
>>
>
>

James Lawton

RE: [SPAM]Re: How to import a pre existing product database

In reply to this post by BJ Freeman

Hmmm, I had appended the logs and a screen shot of what I did - did something strip it out?

Trying again:

Ofbiz.Log snippet from time frame
=======================================================================
2010-12-01 15:15:19,937 (http-0.0.0.0-8443-7) [ ControlServlet.java:327:INFO ] [[[login] Request Done- total:0.073,since last([login] Request B...):0.073]]
2010-12-01 15:15:19,941 (http-0.0.0.0-8443-7) [ ControlServlet.java:144:INFO ] [[[viewdatafile] Request Begun, encoding=[UTF-8]- total:0.0,since last(Begin):0.0]]
2010-12-01 15:15:19,948 (http-0.0.0.0-8443-7) [ RequestHandler.java:740:INFO ] Rendering View [viewdatafile], sessionId=C4503F3026169883A29BCA0B35E92EEF.jvm1
2010-12-01 15:15:19,982 (http-0.0.0.0-8443-7) [ ScreenFactory.java:128:INFO ] Got 2 screens in 0.0070s from: file:/C:/ofbiz/mysqlrelease10.04/framework/webtools/widget/MiscScreens.xml
2010-12-01 15:15:20,233 (http-0.0.0.0-8443-7) [ ServerHitBin.java:627:INFO ] Visit delegatorName=default, ServerHitBin delegatorName=default
2010-12-01 15:15:20,235 (http-0.0.0.0-8443-7) [ ControlServlet.java:327:INFO ] [[[viewdatafile] Request Done- total:0.294,since last([viewdatafile] Re...):0.294]]
2010-12-01 15:15:24,219 (http-0.0.0.0-8443-5) [ ContextFilter.java:201:INFO ] [Request]: /webtools/control/viewdatafile
2010-12-01 15:15:24,219 (http-0.0.0.0-8443-5) [ ControlServlet.java:144:INFO ] [[[viewdatafile] Request Begun, encoding=[UTF-8]- total:0.0,since last(Begin):0.0]]
2010-12-01 15:15:24,229 (http-0.0.0.0-8443-5) [ ConfigXMLReader.java:120:INFO ] controller loaded: 0.0s, 23 requests, 16 views in file:/C:/ofbiz/mysqlrelease10.04/framework/common/webcommon/WEB-INF/common-controller.xml
2010-12-01 15:15:24,233 (http-0.0.0.0-8443-5) [ ConfigXMLReader.java:120:INFO ] controller loaded: 0.0s, 5 requests, 0 views in file:/C:/ofbiz/mysqlrelease10.04/framework/common/webcommon/WEB-INF/tempexpr-controller.xml
2010-12-01 15:15:24,237 (http-0.0.0.0-8443-5) [ ConfigXMLReader.java:120:INFO ] controller loaded: 0.0010s, 8 requests, 2 views in file:/C:/ofbiz/mysqlrelease10.04/framework/testtools/webapp/testtools/WEB-INF/controller.xml
2010-12-01 15:15:24,238 (http-0.0.0.0-8443-5) [ ConfigXMLReader.java:120:INFO ] controller loaded: 0.013s, 110 requests, 64 views in file:/C:/ofbiz/mysqlrelease10.04/framework/webtools/webapp/webtools/WEB-INF/controller.xml
2010-12-01 15:15:24,246 (http-0.0.0.0-8443-5) [ RequestHandler.java:740:INFO ] Rendering View [viewdatafile], sessionId=C4503F3026169883A29BCA0B35E92EEF.jvm1
2010-12-01 15:15:24,294 (http-0.0.0.0-8443-5) [ModelDataFileReader.java:65 :INFO ] [ModelDataFileReader.getModelDataFileReader] : creating reader.
2010-12-01 15:15:24,348 (http-0.0.0.0-8443-5) [ ScreenFactory.java:128:INFO ] Got 10 screens in 0.0070s from: file:/C:/ofbiz/mysqlrelease10.04/framework/webtools/widget/CommonScreens.xml
2010-12-01 15:15:24,358 (http-0.0.0.0-8443-5) [ ScreenFactory.java:128:INFO ] Got 22 screens in 0.0090s from: file:/C:/ofbiz/mysqlrelease10.04/framework/common/widget/CommonScreens.xml
2010-12-01 15:15:24,512 (http-0.0.0.0-8443-5) [ ServerHitBin.java:627:INFO ] Visit delegatorName=default, ServerHitBin delegatorName=default
2010-12-01 15:15:24,515 (http-0.0.0.0-8443-5) [ ControlServlet.java:327:INFO ] [[[viewdatafile] Request Done- total:0.296,since last([viewdatafile] Re...):0.296]]
2010-12-01 15:15:29,084 (http-0.0.0.0-8443-5) [ ContextFilter.java:201:INFO ] [Request]: /webtools/control/viewdatafile
2010-12-01 15:15:29,085 (http-0.0.0.0-8443-5) [ ControlServlet.java:144:INFO ] [[[viewdatafile] Request Begun, encoding=[UTF-8]- total:0.0,since last(Begin):0.0]]
2010-12-01 15:15:29,092 (http-0.0.0.0-8443-5) [ RequestHandler.java:740:INFO ] Rendering View [viewdatafile], sessionId=C4503F3026169883A29BCA0B35E92EEF.jvm1
2010-12-01 15:15:29,134 (http-0.0.0.0-8443-5) [ModelDataFileReader.java:65 :INFO ] [ModelDataFileReader.getModelDataFileReader] : creating reader.
2010-12-01 15:15:29,353 (http-0.0.0.0-8443-5) [ ServerHitBin.java:627:INFO ] Visit delegatorName=default, ServerHitBin delegatorName=default
2010-12-01 15:15:29,357 (http-0.0.0.0-8443-5) [ ControlServlet.java:327:INFO ] [[[viewdatafile] Request Done- total:0.272,since last([viewdatafile] Re...):0.272]]
2010-12-01
=====================================================================================

STeps:
I went to the Webtools / Work with Datafiles screen

I entered the following string in the "Definition Filename or URL" box. The file is attached to the email
"c:\files\tofino\import data\tofinodata.xml"

I selected the "Run" button

Nothing happened and I can't see anything in the ofbiz.log which would point to why the system is not accepting the XML file

Thanks
James

-----Original Message-----
From: BJ Freeman [mailto:[hidden email]]
Sent: December-01-10 3:41 PM
To: [hidden email]
Subject: [SPAM]Re: How to import a pre existing product database

sorry see no logs and not enough detail.
I have no way to replicated your steps.

=========================
BJ Freeman
Strategic Power Office with Supplier Automation <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
Specialtymarket.com <http://www.specialtymarket.com/> Systems Integrator-- Glad to Assist

Chat Y! messenger: bjfr33man

James Lawton sent the following on 12/1/2010 12:23 PM:

> Here is the section of ofbiz.log for the timeframe
>
>
>
>
>
>
>
>
>
> Here is what I did:
>
> entered the file information and pressed "Run"
>
>
>
>
>
>
>
>
>
> Thanks
>
> James
>
>
>
>
>
> -----Original Message-----
> From: BJ Freeman [mailto:[hidden email]]
> Sent: December-01-10 2:18 PM
> To: [hidden email]
> Subject: [SPAM]Re: How to import a pre existing product database
>
>
>
> take a look at your logs.
>
> please give detail steps
>
>
>
>
>
> =========================
>
> BJ Freeman
>
> Strategic Power Office with Supplier Automation<
> <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
> http://www.businessesnetwork.com/automation/viewforum.php?f=52>
>
> Specialtymarket.com< <http://www.specialtymarket.com/>
> http://www.specialtymarket.com/> Systems Integrator-- Glad to Assist
>
>
>
> Chat Y! messenger: bjfr33man
>
>
>
>
>
> James Lawton sent the following on 12/1/2010 10:27 AM:
>
>> Thanks for the link and the record information...
>
>>
>
>> I have altered the fields to fit the product record field names and
>
>> tried to use the "work with data files" tool again
>
>>
>
>> I enter the Definition File Name in the screen
>
>> "<file:///\\locahost\c:\files\tofino\import%20data\tofinodata.xml> file://locahost/c:\files\tofino\import data\tofinodata.xml" and select the URL check box and then hit Run. The screen flickers but nothing happens. The instructions say that a drop down box will appear with a list of the definitions to pick from...
>
>>
>
>> Thoughts?
>
>>
>
>> Here is the modified definition file:
>
>>
>
>> <data-file name="c:\files\tofino\import data\itemdatabasereportmod.txt" separator-style="delimited" type-code="UTF-8"
>
>> delimiter="&#009" text-delimiter=""
>
>> description="this takes the product data from jeffs file"
>
>> has-header="true">
>
>> <record name="Product">
>
>> <field name="manufacturerPartyId" ignored="false"
>> type="String"/>
>
>> <field name="productId" ignored="false" type="String"/>
>
>> <field name="primaryProductCategoryId" expression="true"
>
>> default-value="tools" type="String"/>
>
>> <field name="productTypeId" expression="true"
>
>> default-value="FINISHED_GOOD" type="String"/>
>
>> <field name="productName" ignored="false" type="String"/>
>
>> <field name="description" ignored="false" type="String"/>
>
>> <field name="quantityIncluded" expression="true" default-value="1"
>
>> type="String"/>
>
>> <field name="priceDetailText" ignored="true"
>> type="String"/>
>
>> </record>
>
>> </data-file>
>
>>
>
>>
>
>> Thanks
>
>> James
>
>> -----Original Message-----
>
>> From: BJ Freeman [mailto:[hidden email]]
>
>> Sent: December-01-10 12:37 PM
>
>> To: [hidden email]
>
>> Subject: [SPAM]Re: How to import a pre existing product database
>
>>
>
>> if you look at the table the manufacture field is in then look up the
>
>> same in the entities
>
>> <https://demo-trunk.ofbiz.apache.org/webtools/control/entitymaint>
>> https://demo-trunk.ofbiz.apache.org/webtools/control/entitymaint
>
>> Just to clarify
>
>> your file has a Record=Product.
>
>> this means you will be creating a Product entity file, so only fields in the product entity can be created.
>
>>
>> <https://demo-trunk.ofbiz.apache.org/webtools/control/FindGeneric?ent
>> it>
>> https://demo-trunk.ofbiz.apache.org/webtools/control/FindGeneric?enti
>> t
>
>> yName=Product&find=true&VIEW_SIZE=50&VIEW_INDEX=0
>
>> if you have data that goes in different entities then you need to create a Record for each entity with its fields.
>
>>
>
>>
>
>> James Lawton sent the following on 12/1/2010 9:16 AM:
>
>>
>
>>
>
>> =========================
>
>> BJ Freeman
>
>> Strategic Power Office with Supplier
>
>> Automation<http://www.businessesnetwork.com/automation/viewforum.php?
>> f
>
>> =52> Specialtymarket.com< <http://www.specialtymarket.com/> http://www.specialtymarket.com/> Systems
>
>> Integrator-- Glad to Assist
>
>>
>
>> Chat Y! messenger: bjfr33man
>
>>
>
>>> Thanks
>
>>>
>
>>> I must be missing the spot in web tools it has a complete list of the field names - could you point that out again? For example what is the right field name for the manufacturer / supplier? I did take that out the db....
>
>>>
>
>>> Thanks
>
>>> James
>
>>>
>
>>> -----Original Message-----
>
>>> From: BJ Freeman [mailto:[hidden email]]
>
>>> Sent: November-30-10 11:42 PM
>
>>> To: [hidden email]
>
>>> Subject: [SPAM]Re: How to import a pre existing product database
>
>>>
>
>>> there should be more to the log that would pinpoint what is going on.
>
>>> I am guessing it is your input to the form, since this has not been changed 09.
>
>>> if you look at the file mentioned you will see
>
>>> <script
>
>>> location="component://webtools/webapp/webtools/WEB-INF/actions/dataf
>>> i
>
>>> l
>
>>> e/viewdatafile.groovy"/>
>
>>> that is where the error referenced.
>
>>>
>
>>> your xml(not xsd) should have field names as you see them in webtools.
>
>>> when you have underscores like manufacturer_party_id then that is what is in the db but ofbiz will not recognize it in Webtools or code.
>
>>>
>
>>>
>
>>>
>
>>>
>
>>> James Lawton sent the following on 11/30/2010 7:34 PM:
>
>>>
>
>>>
>
>>>> I think I am getting closer - thanks for the tips and the patience!
>
>>>>
>
>>>> I have been through all the files and here is what I have gotten
>
>>>> when running my definition file through the data tools
>
>>>>
>
>>>> org.ofbiz.widget.screen.ScreenRenderException: Error rendering
>
>>>> screen
>
>>>> [component://webtools/widget/MiscScreens.xml#viewdatafile]:
>
>>>> groovy.lang.MissingPropertyException: No such property:
>
>>>> definitionUrl for class: viewdatafile (No such property: definitionUrl for class:
>
>>>> viewdatafile)
>
>>>>
>
>>>>
>
>>>> Here is the xsd file I was running:
>
>>>>
>
>>>>
>
>>>> <data-file name="itemdatabasereport.txt" separator-style="delimited" type-code="UTF-8"
>
>>>> delimiter="&#009" text-delimiter=""
>
>>>> description="this takes the product data from jeffs file"
>
>>>> has-header="true">
>
>>>> <record name="Product">
>
>>>> <field name="manufacturer_party_id" ignored="false"
>>>> type="String"/>
>
>>>> <field name="productId" ignored="false" type="String"/>
>
>>>> <field name="primaryProductCategoryId" expression="true"
>
>>>> default-value="tools" type="String"/>
>
>>>> <field name="productTypeId" expression="true"
>
>>>> default-value="FINISHED_GOOD" type="String"/>
>
>>>> <field name="productName" ignored="false"
>>>> type="String"/>
>
>>>> <field name="description" ignored="false"
>>>> type="String"/>
>
>>>> <field name="quantityIncluded" expression="true" default-value="1"
>
>>>> type="String"/>
>
>>>> <field name="price" ignored="true" type="String"/>
>
>>>> </record>
>
>>>> </data-file>
>
>>>>
>
>>>>
>
>>>> What's wrong with my definition file which is causing the data tool to throw up?
>
>>>>
>
>>>> Thanks
>
>>>> James
>
>>>>
>
>>>>
>
>>>>
>
>>>> -----Original Message-----
>
>>>> From: BJ Freeman [mailto:[hidden email]]
>
>>>> Sent: November-26-10 1:36 AM
>
>>>> To: [hidden email]
>
>>>> Subject: [SPAM]Re: How to import a pre existing product database
>
>>>>
>
>>>> I am not clear what you are doing.
>
>>>> the example you show is for an entity DataImportProduct which does
>
>>>> not exist in the OOTB ofbiz take a look at
>
>>>>
>
>>>> <https://issues.apache.org/jira/browse/OFBIZ-2534>
>>>> https://issues.apache.org/jira/browse/OFBIZ-2534
>
>>>> for tab examples.
>
>>>> also
>
>>>> <https://issues.apache.org/jira/browse/OFBIZ-2518>
>>>> https://issues.apache.org/jira/browse/OFBIZ-2518
>
>>>>
>
>>>> =========================
>
>>>>
>
>>>> BJ Freeman
>
>>>> Strategic Power Office with Supplier Automation<
>>>> <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
>>>> http://www.businessesnetwork.com/automation/viewforum.php?f=52>
>
>>>> Specialtymarket.com< <http://www.specialtymarket.com/> http://www.specialtymarket.com/> Systems Integrator-- Glad to Assist
>
>>>>
>
>>>> Chat Y! messenger: bjfr33man
>
>>>> James Lawton sent the following on 11/25/2010 6:36 PM:
>
>>>>> I am probably doing something stupid here...I have dug into the
>
>>>>> link that was posted and it looks simple enough...however the code
>
>>>>> snippet is for a fixed width file rather than a comma delimited
>
>>>>> file...and my attempts to create the right schema for importing
>
>>>>> product are failing...If someone has a CSV file snippet to share
>
>>>>> that would be awesome
>
>>>>>
>
>>>>> So heading in a different direction I created the entity data snippet below:
>
>>>>>
>
>>>>>  <entity-engine-xml>
>
>>>>>
>
>>>>> <DataImportProduct productId="YG0081ATF"
>>>>> productTypeId="FINISHED_GOOD" description="1/8 x 1-59/64 CARBIDE
>>>>> DREAM DRILL W/O COOLANT HOLES (3XD)" price="22.13"
>>>>> priceCurrencyUomId="USD" supplierPartyId="YG1" purchasePrice=""/>
>
>>>>> <DataImportProduct productId="YG0081BTF"
>
>>>>> productTypeId="FINISHED_GOOD" description="1/8 x 15/64 CARBIDE
>
>>>>> DREAM DRILL W/ COOLANT HOLE" price="105.73" priceCurrencyUomId="USD"
>
>>>>> supplierPartyId="YG1" purchasePrice=""/>
>
>>>>>
>
>>>>>
>
>>>>> </entity-engine-xml>
>
>>>>>
>
>>>>> However every time I have tried to use the XML IMport tool it just
>
>>>>> keeps coming back and saying that the either URL / file or the XML
>
>>>>> are not complete
>
>>>>>
>
>>>>> What am I doing wrong?
>
>>>>>
>
>>>>> -----Original Message-----
>
>>>>> From: BJ Freeman [mailto:[hidden email]]
>
>>>>> Sent: November-09-10 6:00 PM
>
>>>>> To: [hidden email]
>
>>>>> Subject: [SPAM]Re: How to import a pre existing product database
>
>>>>>
>
>>>>> all web tools imports into memory so is limited to the amount of free memory of the sever.
>
>>>>>
>
>>>>> I imports files that are gigs in size so read in the files. I typical file has 25 columns and takes about 15min.
>
>>>>> There is no need to write code to convert files to xml since the
>
>>>>>
>>>>> <https://cwiki.apache.org/confluence/display/OFBENDUSER/OFBiz%27s+
>>>>> Da>
>>>>> https://cwiki.apache.org/confluence/display/OFBENDUSER/OFBiz%27s+D
>>>>> a
>
>>>>> t
>
>>>>> a+
>
>>>>> File+Tools
>
>>>>>
>
>>>>>
>
>>>>>
>
>>>>>
>
>>>>>
>
>>>>> =========================
>
>>>>> BJ Freeman
>
>>>>> Strategic Power Office with Supplier
>
>>>>> Automation<http://www.businessesnetwork.com/automation/viewforum.p
>>>>> hp?f
>
>>>>> =52> Specialtymarket.com< <http://www.specialtymarket.com/> http://www.specialtymarket.com/>
>
>>>>> Systems Integrator-- Glad to Assist
>
>>>>>
>
>>>>> Chat Y! messenger: bjfr33man
>
>>>>> James McGill sent the following on 11/9/2010 12:38 PM:
>
>>>>>
>
>>>>>> On Tue, Nov 9, 2010 at 9:34 AM, James Lawton< <mailto:[hidden email]> [hidden email]>wrote:
>
>>>>>>
>
>>>>>>> How would I import a pre existing product database? I currently
>
>>>>>>> have it in a CSV and I am looking for a method of getting that
>
>>>>>>> information into the database so that the imported items will
>
>>>>>>> show up in the catalog
>
>>>>>>>
>
>>>>>>
>
>>>>>>
>
>>>>>> OFBiz ETL is best done as XML seed data.
>
>>>>>> In your scenario I would seriously consider converting the CSV
>
>>>>>> into XML (a job for Perl), and perhaps doing some input
>>>>>> validation
>
>>>>>> as part of that conversion, and then loading the XML as seed data.
>
>>>>>>
>
>>>>>> If you really have a lot of ETL to do (e.g., processing that will
>
>>>>>> take hours or days to run through the service interface) you
>>>>>> might
>
>>>>>> be better off going straight to the database. You lose some of
>
>>>>>> the traceability this way (e.g., logging and event side-effects)
>
>>>>>> so it isn't recommended. (We ran into problems when loading in
>
>>>>>> several years worth of historical data, orders and inventory
>
>>>>>> transactions.)
>
>>>>>>
>
>>>>>> --
>
>>>>>> James McGill
>
>>>>>> Phoenix AZ
>
>>>>>>
>
>>>>>
>
>>>>>
>
>>>>>
>
>>>>
>
>>>>
>
>>>>
>
>>>
>
>>>
>
>>>
>
>>
>
>>
>
>>
>
>

Tofinodata.xml (1K) Download Attachment

BJ Freeman

Re: How to import a pre existing product database

put the file under the ofbiz home then just the path from the ofbiz_home
where you put it. do not check URL.
the click on run
you should see an entry in
Data File Definition Name
for each record you have defined in the file.

=========================
BJ Freeman
Strategic Power Office with Supplier Automation <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
Specialtymarket.com <http://www.specialtymarket.com/>
Systems Integrator-- Glad to Assist

Chat Y! messenger: bjfr33man

James Lawton sent the following on 12/1/2010 2:43 PM:

> Hmmm, I had appended the logs and a screen shot of what I did - did something strip it out?
>
> Trying again:
>
> Ofbiz.Log snippet from time frame
> =======================================================================
> 2010-12-01 15:15:19,937 (http-0.0.0.0-8443-7) [ ControlServlet.java:327:INFO ] [[[login] Request Done- total:0.073,since last([login] Request B...):0.073]]
> 2010-12-01 15:15:19,941 (http-0.0.0.0-8443-7) [ ControlServlet.java:144:INFO ] [[[viewdatafile] Request Begun, encoding=[UTF-8]- total:0.0,since last(Begin):0.0]]
> 2010-12-01 15:15:19,948 (http-0.0.0.0-8443-7) [ RequestHandler.java:740:INFO ] Rendering View [viewdatafile], sessionId=C4503F3026169883A29BCA0B35E92EEF.jvm1
> 2010-12-01 15:15:19,982 (http-0.0.0.0-8443-7) [ ScreenFactory.java:128:INFO ] Got 2 screens in 0.0070s from: file:/C:/ofbiz/mysqlrelease10.04/framework/webtools/widget/MiscScreens.xml
> 2010-12-01 15:15:20,233 (http-0.0.0.0-8443-7) [ ServerHitBin.java:627:INFO ] Visit delegatorName=default, ServerHitBin delegatorName=default
> 2010-12-01 15:15:20,235 (http-0.0.0.0-8443-7) [ ControlServlet.java:327:INFO ] [[[viewdatafile] Request Done- total:0.294,since last([viewdatafile] Re...):0.294]]
> 2010-12-01 15:15:24,219 (http-0.0.0.0-8443-5) [ ContextFilter.java:201:INFO ] [Request]: /webtools/control/viewdatafile
> 2010-12-01 15:15:24,219 (http-0.0.0.0-8443-5) [ ControlServlet.java:144:INFO ] [[[viewdatafile] Request Begun, encoding=[UTF-8]- total:0.0,since last(Begin):0.0]]
> 2010-12-01 15:15:24,229 (http-0.0.0.0-8443-5) [ ConfigXMLReader.java:120:INFO ] controller loaded: 0.0s, 23 requests, 16 views in file:/C:/ofbiz/mysqlrelease10.04/framework/common/webcommon/WEB-INF/common-controller.xml
> 2010-12-01 15:15:24,233 (http-0.0.0.0-8443-5) [ ConfigXMLReader.java:120:INFO ] controller loaded: 0.0s, 5 requests, 0 views in file:/C:/ofbiz/mysqlrelease10.04/framework/common/webcommon/WEB-INF/tempexpr-controller.xml
> 2010-12-01 15:15:24,237 (http-0.0.0.0-8443-5) [ ConfigXMLReader.java:120:INFO ] controller loaded: 0.0010s, 8 requests, 2 views in file:/C:/ofbiz/mysqlrelease10.04/framework/testtools/webapp/testtools/WEB-INF/controller.xml
> 2010-12-01 15:15:24,238 (http-0.0.0.0-8443-5) [ ConfigXMLReader.java:120:INFO ] controller loaded: 0.013s, 110 requests, 64 views in file:/C:/ofbiz/mysqlrelease10.04/framework/webtools/webapp/webtools/WEB-INF/controller.xml
> 2010-12-01 15:15:24,246 (http-0.0.0.0-8443-5) [ RequestHandler.java:740:INFO ] Rendering View [viewdatafile], sessionId=C4503F3026169883A29BCA0B35E92EEF.jvm1
> 2010-12-01 15:15:24,294 (http-0.0.0.0-8443-5) [ModelDataFileReader.java:65 :INFO ] [ModelDataFileReader.getModelDataFileReader] : creating reader.
> 2010-12-01 15:15:24,348 (http-0.0.0.0-8443-5) [ ScreenFactory.java:128:INFO ] Got 10 screens in 0.0070s from: file:/C:/ofbiz/mysqlrelease10.04/framework/webtools/widget/CommonScreens.xml
> 2010-12-01 15:15:24,358 (http-0.0.0.0-8443-5) [ ScreenFactory.java:128:INFO ] Got 22 screens in 0.0090s from: file:/C:/ofbiz/mysqlrelease10.04/framework/common/widget/CommonScreens.xml
> 2010-12-01 15:15:24,512 (http-0.0.0.0-8443-5) [ ServerHitBin.java:627:INFO ] Visit delegatorName=default, ServerHitBin delegatorName=default
> 2010-12-01 15:15:24,515 (http-0.0.0.0-8443-5) [ ControlServlet.java:327:INFO ] [[[viewdatafile] Request Done- total:0.296,since last([viewdatafile] Re...):0.296]]
> 2010-12-01 15:15:29,084 (http-0.0.0.0-8443-5) [ ContextFilter.java:201:INFO ] [Request]: /webtools/control/viewdatafile
> 2010-12-01 15:15:29,085 (http-0.0.0.0-8443-5) [ ControlServlet.java:144:INFO ] [[[viewdatafile] Request Begun, encoding=[UTF-8]- total:0.0,since last(Begin):0.0]]
> 2010-12-01 15:15:29,092 (http-0.0.0.0-8443-5) [ RequestHandler.java:740:INFO ] Rendering View [viewdatafile], sessionId=C4503F3026169883A29BCA0B35E92EEF.jvm1
> 2010-12-01 15:15:29,134 (http-0.0.0.0-8443-5) [ModelDataFileReader.java:65 :INFO ] [ModelDataFileReader.getModelDataFileReader] : creating reader.
> 2010-12-01 15:15:29,353 (http-0.0.0.0-8443-5) [ ServerHitBin.java:627:INFO ] Visit delegatorName=default, ServerHitBin delegatorName=default
> 2010-12-01 15:15:29,357 (http-0.0.0.0-8443-5) [ ControlServlet.java:327:INFO ] [[[viewdatafile] Request Done- total:0.272,since last([viewdatafile] Re...):0.272]]
> 2010-12-01
> =====================================================================================
>
>
> STeps:
> I went to the Webtools / Work with Datafiles screen
>
> I entered the following string in the "Definition Filename or URL" box. The file is attached to the email
> "c:\files\tofino\import data\tofinodata.xml"
>
> I selected the "Run" button
>
> Nothing happened and I can't see anything in the ofbiz.log which would point to why the system is not accepting the XML file
>
> Thanks
> James
>
> -----Original Message-----
> From: BJ Freeman [mailto:[hidden email]]
> Sent: December-01-10 3:41 PM
> To: [hidden email]
> Subject: [SPAM]Re: How to import a pre existing product database
>
> sorry see no logs and not enough detail.
> I have no way to replicated your steps.
>
> =========================
> BJ Freeman
> Strategic Power Office with Supplier Automation<http://www.businessesnetwork.com/automation/viewforum.php?f=52>
> Specialtymarket.com<http://www.specialtymarket.com/> Systems Integrator-- Glad to Assist
>
> Chat Y! messenger: bjfr33man
>
>
> James Lawton sent the following on 12/1/2010 12:23 PM:
>> Here is the section of ofbiz.log for the timeframe
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> Here is what I did:
>>
>> entered the file information and pressed "Run"
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> Thanks
>>
>> James
>>
>>
>>
>>
>>
>> -----Original Message-----
>> From: BJ Freeman [mailto:[hidden email]]
>> Sent: December-01-10 2:18 PM
>> To: [hidden email]
>> Subject: [SPAM]Re: How to import a pre existing product database
>>
>>
>>
>> take a look at your logs.
>>
>> please give detail steps
>>
>>
>>
>>
>>
>> =========================
>>
>> BJ Freeman
>>
>> Strategic Power Office with Supplier Automation<
>> <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
>> http://www.businessesnetwork.com/automation/viewforum.php?f=52>
>>
>> Specialtymarket.com< <http://www.specialtymarket.com/>
>> http://www.specialtymarket.com/> Systems Integrator-- Glad to Assist
>>
>>
>>
>> Chat Y! messenger: bjfr33man
>>
>>
>>
>>
>>
>> James Lawton sent the following on 12/1/2010 10:27 AM:
>>
>>> Thanks for the link and the record information...
>>
>>>
>>
>>> I have altered the fields to fit the product record field names and
>>
>>> tried to use the "work with data files" tool again
>>
>>>
>>
>>> I enter the Definition File Name in the screen
>>
>>> "<file:///\\locahost\c:\files\tofino\import%20data\tofinodata.xml> file://locahost/c:\files\tofino\import data\tofinodata.xml" and select the URL check box and then hit Run. The screen flickers but nothing happens. The instructions say that a drop down box will appear with a list of the definitions to pick from...
>>
>>>
>>
>>> Thoughts?
>>
>>>
>>
>>> Here is the modified definition file:
>>
>>>
>>
>>> <data-file name="c:\files\tofino\import data\itemdatabasereportmod.txt" separator-style="delimited" type-code="UTF-8"
>>
>>> delimiter="&#009" text-delimiter=""
>>
>>> description="this takes the product data from jeffs file"
>>
>>> has-header="true">
>>
>>> <record name="Product">
>>
>>> <field name="manufacturerPartyId" ignored="false"
>>> type="String"/>
>>
>>> <field name="productId" ignored="false" type="String"/>
>>
>>> <field name="primaryProductCategoryId" expression="true"
>>
>>> default-value="tools" type="String"/>
>>
>>> <field name="productTypeId" expression="true"
>>
>>> default-value="FINISHED_GOOD" type="String"/>
>>
>>> <field name="productName" ignored="false" type="String"/>
>>
>>> <field name="description" ignored="false" type="String"/>
>>
>>> <field name="quantityIncluded" expression="true" default-value="1"
>>
>>> type="String"/>
>>
>>> <field name="priceDetailText" ignored="true"
>>> type="String"/>
>>
>>> </record>
>>
>>> </data-file>
>>
>>>
>>
>>>
>>
>>> Thanks
>>
>>> James
>>
>>> -----Original Message-----
>>
>>> From: BJ Freeman [mailto:[hidden email]]
>>
>>> Sent: December-01-10 12:37 PM
>>
>>> To: [hidden email]
>>
>>> Subject: [SPAM]Re: How to import a pre existing product database
>>
>>>
>>
>>> if you look at the table the manufacture field is in then look up the
>>
>>> same in the entities
>>
>>> <https://demo-trunk.ofbiz.apache.org/webtools/control/entitymaint>
>>> https://demo-trunk.ofbiz.apache.org/webtools/control/entitymaint
>>
>>> Just to clarify
>>
>>> your file has a Record=Product.
>>
>>> this means you will be creating a Product entity file, so only fields in the product entity can be created.
>>
>>>
>>> <https://demo-trunk.ofbiz.apache.org/webtools/control/FindGeneric?ent
>>> it>
>>> https://demo-trunk.ofbiz.apache.org/webtools/control/FindGeneric?enti
>>> t
>>
>>> yName=Product&find=true&VIEW_SIZE=50&VIEW_INDEX=0
>>
>>> if you have data that goes in different entities then you need to create a Record for each entity with its fields.
>>
>>>
>>
>>>
>>
>>> James Lawton sent the following on 12/1/2010 9:16 AM:
>>
>>>
>>
>>>
>>
>>> =========================
>>
>>> BJ Freeman
>>
>>> Strategic Power Office with Supplier
>>
>>> Automation<http://www.businessesnetwork.com/automation/viewforum.php?
>>> f
>>
>>> =52> Specialtymarket.com< <http://www.specialtymarket.com/> http://www.specialtymarket.com/> Systems
>>
>>> Integrator-- Glad to Assist
>>
>>>
>>
>>> Chat Y! messenger: bjfr33man
>>
>>>
>>
>>>> Thanks
>>
>>>>
>>
>>>> I must be missing the spot in web tools it has a complete list of the field names - could you point that out again? For example what is the right field name for the manufacturer / supplier? I did take that out the db....
>>
>>>>
>>
>>>> Thanks
>>
>>>> James
>>
>>>>
>>
>>>> -----Original Message-----
>>
>>>> From: BJ Freeman [mailto:[hidden email]]
>>
>>>> Sent: November-30-10 11:42 PM
>>
>>>> To: [hidden email]
>>
>>>> Subject: [SPAM]Re: How to import a pre existing product database
>>
>>>>
>>
>>>> there should be more to the log that would pinpoint what is going on.
>>
>>>> I am guessing it is your input to the form, since this has not been changed 09.
>>
>>>> if you look at the file mentioned you will see
>>
>>>> <script
>>
>>>> location="component://webtools/webapp/webtools/WEB-INF/actions/dataf
>>>> i
>>
>>>> l
>>
>>>> e/viewdatafile.groovy"/>
>>
>>>> that is where the error referenced.
>>
>>>>
>>
>>>> your xml(not xsd) should have field names as you see them in webtools.
>>
>>>> when you have underscores like manufacturer_party_id then that is what is in the db but ofbiz will not recognize it in Webtools or code.
>>
>>>>
>>
>>>>
>>
>>>>
>>
>>>>
>>
>>>> James Lawton sent the following on 11/30/2010 7:34 PM:
>>
>>>>
>>
>>>>
>>
>>>>> I think I am getting closer - thanks for the tips and the patience!
>>
>>>>>
>>
>>>>> I have been through all the files and here is what I have gotten
>>
>>>>> when running my definition file through the data tools
>>
>>>>>
>>
>>>>> org.ofbiz.widget.screen.ScreenRenderException: Error rendering
>>
>>>>> screen
>>
>>>>> [component://webtools/widget/MiscScreens.xml#viewdatafile]:
>>
>>>>> groovy.lang.MissingPropertyException: No such property:
>>
>>>>> definitionUrl for class: viewdatafile (No such property: definitionUrl for class:
>>
>>>>> viewdatafile)
>>
>>>>>
>>
>>>>>
>>
>>>>> Here is the xsd file I was running:
>>
>>>>>
>>
>>>>>
>>
>>>>> <data-file name="itemdatabasereport.txt" separator-style="delimited" type-code="UTF-8"
>>
>>>>> delimiter="&#009" text-delimiter=""
>>
>>>>> description="this takes the product data from jeffs file"
>>
>>>>> has-header="true">
>>
>>>>> <record name="Product">
>>
>>>>> <field name="manufacturer_party_id" ignored="false"
>>>>> type="String"/>
>>
>>>>> <field name="productId" ignored="false" type="String"/>
>>
>>>>> <field name="primaryProductCategoryId" expression="true"
>>
>>>>> default-value="tools" type="String"/>
>>
>>>>> <field name="productTypeId" expression="true"
>>
>>>>> default-value="FINISHED_GOOD" type="String"/>
>>
>>>>> <field name="productName" ignored="false"
>>>>> type="String"/>
>>
>>>>> <field name="description" ignored="false"
>>>>> type="String"/>
>>
>>>>> <field name="quantityIncluded" expression="true" default-value="1"
>>
>>>>> type="String"/>
>>
>>>>> <field name="price" ignored="true" type="String"/>
>>
>>>>> </record>
>>
>>>>> </data-file>
>>
>>>>>
>>
>>>>>
>>
>>>>> What's wrong with my definition file which is causing the data tool to throw up?
>>
>>>>>
>>
>>>>> Thanks
>>
>>>>> James
>>
>>>>>
>>
>>>>>
>>
>>>>>
>>
>>>>> -----Original Message-----
>>
>>>>> From: BJ Freeman [mailto:[hidden email]]
>>
>>>>> Sent: November-26-10 1:36 AM
>>
>>>>> To: [hidden email]
>>
>>>>> Subject: [SPAM]Re: How to import a pre existing product database
>>
>>>>>
>>
>>>>> I am not clear what you are doing.
>>
>>>>> the example you show is for an entity DataImportProduct which does
>>
>>>>> not exist in the OOTB ofbiz take a look at
>>
>>>>>
>>
>>>>> <https://issues.apache.org/jira/browse/OFBIZ-2534>
>>>>> https://issues.apache.org/jira/browse/OFBIZ-2534
>>
>>>>> for tab examples.
>>
>>>>> also
>>
>>>>> <https://issues.apache.org/jira/browse/OFBIZ-2518>
>>>>> https://issues.apache.org/jira/browse/OFBIZ-2518
>>
>>>>>
>>
>>>>> =========================
>>
>>>>>
>>
>>>>> BJ Freeman
>>
>>>>> Strategic Power Office with Supplier Automation<
>>>>> <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
>>>>> http://www.businessesnetwork.com/automation/viewforum.php?f=52>
>>
>>>>> Specialtymarket.com< <http://www.specialtymarket.com/> http://www.specialtymarket.com/> Systems Integrator-- Glad to Assist
>>
>>>>>
>>
>>>>> Chat Y! messenger: bjfr33man
>>
>>>>> James Lawton sent the following on 11/25/2010 6:36 PM:
>>
>>>>>> I am probably doing something stupid here...I have dug into the
>>
>>>>>> link that was posted and it looks simple enough...however the code
>>
>>>>>> snippet is for a fixed width file rather than a comma delimited
>>
>>>>>> file...and my attempts to create the right schema for importing
>>
>>>>>> product are failing...If someone has a CSV file snippet to share
>>
>>>>>> that would be awesome
>>
>>>>>>
>>
>>>>>> So heading in a different direction I created the entity data snippet below:
>>
>>>>>>
>>
>>>>>>  <entity-engine-xml>
>>
>>>>>>
>>
>>>>>> <DataImportProduct productId="YG0081ATF"
>>>>>> productTypeId="FINISHED_GOOD" description="1/8 x 1-59/64 CARBIDE
>>>>>> DREAM DRILL W/O COOLANT HOLES (3XD)" price="22.13"
>>>>>> priceCurrencyUomId="USD" supplierPartyId="YG1" purchasePrice=""/>
>>
>>>>>> <DataImportProduct productId="YG0081BTF"
>>
>>>>>> productTypeId="FINISHED_GOOD" description="1/8 x 15/64 CARBIDE
>>
>>>>>> DREAM DRILL W/ COOLANT HOLE" price="105.73" priceCurrencyUomId="USD"
>>
>>>>>> supplierPartyId="YG1" purchasePrice=""/>
>>
>>>>>>
>>
>>>>>>
>>
>>>>>> </entity-engine-xml>
>>
>>>>>>
>>
>>>>>> However every time I have tried to use the XML IMport tool it just
>>
>>>>>> keeps coming back and saying that the either URL / file or the XML
>>
>>>>>> are not complete
>>
>>>>>>
>>
>>>>>> What am I doing wrong?
>>
>>>>>>
>>
>>>>>> -----Original Message-----
>>
>>>>>> From: BJ Freeman [mailto:[hidden email]]
>>
>>>>>> Sent: November-09-10 6:00 PM
>>
>>>>>> To: [hidden email]
>>
>>>>>> Subject: [SPAM]Re: How to import a pre existing product database
>>
>>>>>>
>>
>>>>>> all web tools imports into memory so is limited to the amount of free memory of the sever.
>>
>>>>>>
>>
>>>>>> I imports files that are gigs in size so read in the files. I typical file has 25 columns and takes about 15min.
>>
>>>>>> There is no need to write code to convert files to xml since the
>>
>>>>>>
>>>>>> <https://cwiki.apache.org/confluence/display/OFBENDUSER/OFBiz%27s+
>>>>>> Da>
>>>>>> https://cwiki.apache.org/confluence/display/OFBENDUSER/OFBiz%27s+D
>>>>>> a
>>
>>>>>> t
>>
>>>>>> a+
>>
>>>>>> File+Tools
>>
>>>>>>
>>
>>>>>>
>>
>>>>>>
>>
>>>>>>
>>
>>>>>>
>>
>>>>>> =========================
>>
>>>>>> BJ Freeman
>>
>>>>>> Strategic Power Office with Supplier
>>
>>>>>> Automation<http://www.businessesnetwork.com/automation/viewforum.p
>>>>>> hp?f
>>
>>>>>> =52> Specialtymarket.com< <http://www.specialtymarket.com/> http://www.specialtymarket.com/>
>>
>>>>>> Systems Integrator-- Glad to Assist
>>
>>>>>>
>>
>>>>>> Chat Y! messenger: bjfr33man
>>
>>>>>> James McGill sent the following on 11/9/2010 12:38 PM:
>>
>>>>>>
>>
>>>>>>> On Tue, Nov 9, 2010 at 9:34 AM, James Lawton< <mailto:[hidden email]> [hidden email]>wrote:
>>
>>>>>>>
>>
>>>>>>>> How would I import a pre existing product database? I currently
>>
>>>>>>>> have it in a CSV and I am looking for a method of getting that
>>
>>>>>>>> information into the database so that the imported items will
>>
>>>>>>>> show up in the catalog
>>
>>>>>>>>
>>
>>>>>>>
>>
>>>>>>>
>>
>>>>>>> OFBiz ETL is best done as XML seed data.
>>
>>>>>>> In your scenario I would seriously consider converting the CSV
>>
>>>>>>> into XML (a job for Perl), and perhaps doing some input
>>>>>>> validation
>>
>>>>>>> as part of that conversion, and then loading the XML as seed data.
>>
>>>>>>>
>>
>>>>>>> If you really have a lot of ETL to do (e.g., processing that will
>>
>>>>>>> take hours or days to run through the service interface) you
>>>>>>> might
>>
>>>>>>> be better off going straight to the database. You lose some of
>>
>>>>>>> the traceability this way (e.g., logging and event side-effects)
>>
>>>>>>> so it isn't recommended. (We ran into problems when loading in
>>
>>>>>>> several years worth of historical data, orders and inventory
>>
>>>>>>> transactions.)
>>
>>>>>>>
>>
>>>>>>> --
>>
>>>>>>> James McGill
>>
>>>>>>> Phoenix AZ
>>
>>>>>>>
>>
>>>>>>
>>
>>>>>>
>>
>>>>>>
>>
>>>>>
>>
>>>>>
>>
>>>>>
>>
>>>>
>>
>>>>
>>
>>>>
>>
>>>
>>
>>>
>>
>>>
>>
>>

James Lawton

RE: How to import a pre existing product database

I am running Windows 7

My ofbiz instance is running in c:\ofbiz\mysqlrelease10.04 - which is where I run "startofbiz.bat" from...

I have placed the file I had attached earlier, tofinodata.xml, into the following directories one after another and did not check URL:
c:\ofbiz - in this case I just used the "tofinodata.xml" on the assumption this could be considered ofbiz_home
c:\ofbiz\mysqlrelease10.04 - in this case I just the "tofinodata.xml" on the assumption this could be considered ofbiz_home
c:\ofbiz\mysqlrelease10.04\import - in this case I used "import\tofinodata.xml"

Nothing happened in all three instances and there is nothing in ofbiz.log about not finding the file

Is there a problem with the contents of the file itself? I have only defined one record at the moment to control the variables in this equation
I am attaching the file again in the hopes that there is a glaring error in it.

Thanks
James

-----Original Message-----
From: BJ Freeman [mailto:[hidden email]]
Sent: December-01-10 7:11 PM
To: [hidden email]
Subject: Re: How to import a pre existing product database

put the file under the ofbiz home then just the path from the ofbiz_home where you put it. do not check URL.
the click on run
you should see an entry in
Data File Definition Name
for each record you have defined in the file.

=========================
BJ Freeman
Strategic Power Office with Supplier Automation <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
Specialtymarket.com <http://www.specialtymarket.com/>
Systems Integrator-- Glad to Assist

Chat Y! messenger: bjfr33man

James Lawton sent the following on 12/1/2010 2:43 PM:

Tofinodata.xml (1K) Download Attachment