DATA IMPORT

[James Troup-4]

Hi All,
I work for a small company in Adelaide Australia and we are in the
process of getting OfBiz up and running.

I am facing the daunting task of getting the historical data out of the
old unix based system we are using into OfBiz.

I have looked through the info in the wiki on the go-integral.com
website, however I am still feeling quite under equipped for the task.

Can anyone point me to some more documentation on importing data into OfBiz?

Thanks,
James

[Brett]

James,


A lot will depend on how you are mapping your data to the ofbiz
tables/entities.  Once you have the mapping you can export the data
into an XML document that matches the <entity-engine-xml>
</entity-engine-xml> format.  Then you can use the ofbiz webtools
application to import the data.  You can look at the
ecommerce/data/Demo*.xml files for examples.

The difficult part is going to be mapping the data correctly.  There
are a lot of foreign key constraints that you must adhere to or the
imports will fail.

Let me know if you need more details.


Brett


On 10/13/06, James Troup <[hidden email]> wrote:

> Hi All,
> I work for a small company in Adelaide Australia and we are in the
> process of getting OfBiz up and running.
>
> I am facing the daunting task of getting the historical data out of the
> old unix based system we are using into OfBiz.
>
> I have looked through the info in the wiki on the go-integral.com
> website, however I am still feeling quite under equipped for the task.
>
> Can anyone point me to some more documentation on importing data into OfBiz?
>
> Thanks,
> James
>

[BJ Freeman]

How are you going to access the orginal data?

James Troup sent the following on 10/13/2006 6:39 PM:

> Hi All,
> I work for a small company in Adelaide Australia and we are in the
> process of getting OfBiz up and running.
>
> I am facing the daunting task of getting the historical data out of the
> old unix based system we are using into OfBiz.
>
> I have looked through the info in the wiki on the go-integral.com
> website, however I am still feeling quite under equipped for the task.
>
> Can anyone point me to some more documentation on importing data into
> OfBiz?
>
> Thanks,
> James
>

[James Troup-4]

Hi BJ,
Original data can be spooled out as fixed width plain text as it
currently prints to tractor feed paper.
Otherwise, I am told there is an ODBC interface (but no one has figured
out how to use it).
Obviously the ODBC interface would be nice to plugin to, but I don't
know if it is more hassle than what it is worth.

James

BJ Freeman wrote:

> How are you going to access the orginal data?
>
> James Troup sent the following on 10/13/2006 6:39 PM:
>> Hi All,
>> I work for a small company in Adelaide Australia and we are in the
>> process of getting OfBiz up and running.
>>
>> I am facing the daunting task of getting the historical data out of
>> the old unix based system we are using into OfBiz.
>>
>> I have looked through the info in the wiki on the go-integral.com
>> website, however I am still feeling quite under equipped for the task.
>>
>> Can anyone point me to some more documentation on importing data into
>> OfBiz?
>>
>> Thanks,
>> James
>>

[BJ Freeman]

based on experience, it is better to have a direct connect, based on
manhours to get fixed width formatted and verified.

do a goggle search on
Unix odbc java
and you will get a lot of links
this one has many providers
http://www.unixodbc.org/drivers.html

 From there you have a couple of options.
one is to just dump the data and manually massage it to ofbiZ data. Not
recommenced.
second is to connect it to ofbiZ as it own instances and then write some
  code to port from it to ofbiz.
third is to write entity defs for the old data then use the entity
engine to covert from the old data to the new data.

each has it pluses and minuses.


James Troup sent the following on 10/13/2006 11:57 PM:

> Hi BJ,
> Original data can be spooled out as fixed width plain text as it
> currently prints to tractor feed paper.
> Otherwise, I am told there is an ODBC interface (but no one has figured
> out how to use it).
> Obviously the ODBC interface would be nice to plugin to, but I don't
> know if it is more hassle than what it is worth.
>
> James
>
> BJ Freeman wrote:
>> How are you going to access the orginal data?
>>
>> James Troup sent the following on 10/13/2006 6:39 PM:
>>> Hi All,
>>> I work for a small company in Adelaide Australia and we are in the
>>> process of getting OfBiz up and running.
>>>
>>> I am facing the daunting task of getting the historical data out of
>>> the old unix based system we are using into OfBiz.
>>>
>>> I have looked through the info in the wiki on the go-integral.com
>>> website, however I am still feeling quite under equipped for the task.
>>>
>>> Can anyone point me to some more documentation on importing data into
>>> OfBiz?
>>>
>>> Thanks,
>>> James
>>>
>

[Walter Vaughan]

James Troup wrote:

> Hi BJ,
> Original data can be spooled out as fixed width plain text as it
> currently prints to tractor feed paper.
> Otherwise, I am told there is an ODBC interface (but no one has figured
> out how to use it).
> Obviously the ODBC interface would be nice to plugin to, but I don't
> know if it is more hassle than what it is worth.

Any chance your unix system is filePro based?

Regardless, daunting is a putting it mildly.

The fact that you said you can only control the data via fixed reports that you
intercept as text files is only going to make this harder.

We've spent months on importing data. There is no magic import pill.

What's your time frame for conversion?

Are you or is are the people on your team Java programmers?

Do you really need to bring in all your historical information, or could you get
by with just product information, and customer list? And keep the old system up
for years?

--
Walter

[James Troup-4]

The unix system is Acucobol.

So are you suggesting that getting all historical data is a very tall order?

I would have thought that once you have mapped the chart of accounts,
creditors and debtors, you could then dump the transaction journal?

Or am I missing the point here?

Walter Vaughan wrote:

> James Troup wrote:
>
>> Hi BJ,
>> Original data can be spooled out as fixed width plain text as it
>> currently prints to tractor feed paper.
>> Otherwise, I am told there is an ODBC interface (but no one has
>> figured out how to use it).
>> Obviously the ODBC interface would be nice to plugin to, but I don't
>> know if it is more hassle than what it is worth.
>
> Any chance your unix system is filePro based?
>
> Regardless, daunting is a putting it mildly.
>
> The fact that you said you can only control the data via fixed reports
> that you intercept as text files is only going to make this harder.
>
> We've spent months on importing data. There is no magic import pill.
>
> What's your time frame for conversion?
>
> Are you or is are the people on your team Java programmers?
>
> Do you really need to bring in all your historical information, or
> could you get by with just product information, and customer list? And
> keep the old system up for years?
>
> --
> Walter

--
        *LTS Australia
32 Cavendish Ave
Devon Park SA 5008*
*P: [phone]
M: 0433 207 447
F: **[phone]*
*E: [hidden email]*

[David E Jones-2]

Data mapping always seems to end up being far more complicated than  
it appears at the start. That may be what people have been referring  
to as the difficult part of this...

Tools can sometimes help with this, but only so much.

If you are exported to flat files then the tool you'll want to use in  
OFBiz to read in that data is the Datafile tool, which can read flat  
files into generic structures, and right them out again from the same  
generic structures. The flat file format and fields definitions and  
such are specified using XML files which makes code based on it more  
flexible and easier to work with as understanding, and perhaps  
requirements, change. With this you can write Java code to map the  
incoming data from DataFile objects to GenericValue objects that the  
Entity Engine can then drop into the database for you. You might want  
to consider using a scripting language (BeanShell is well supported  
in OFBiz) to make this a little easier to write/test/change/test/etc.

-David


On Oct 14, 2006, at 10:09 PM, James Troup wrote:

> The unix system is Acucobol.
>
> So are you suggesting that getting all historical data is a very  
> tall order?
>
> I would have thought that once you have mapped the chart of  
> accounts, creditors and debtors, you could then dump the  
> transaction journal?
>
> Or am I missing the point here?
>
> Walter Vaughan wrote:
>> James Troup wrote:
>>
>>> Hi BJ,
>>> Original data can be spooled out as fixed width plain text as it  
>>> currently prints to tractor feed paper.
>>> Otherwise, I am told there is an ODBC interface (but no one has  
>>> figured out how to use it).
>>> Obviously the ODBC interface would be nice to plugin to, but I  
>>> don't know if it is more hassle than what it is worth.
>>
>> Any chance your unix system is filePro based?
>>
>> Regardless, daunting is a putting it mildly.
>>
>> The fact that you said you can only control the data via fixed  
>> reports that you intercept as text files is only going to make  
>> this harder.
>>
>> We've spent months on importing data. There is no magic import pill.
>>
>> What's your time frame for conversion?
>>
>> Are you or is are the people on your team Java programmers?
>>
>> Do you really need to bring in all your historical information, or  
>> could you get by with just product information, and customer list?  
>> And keep the old system up for years?
>>
>> --
>> Walter
>
> --
> *LTS Australia
> 32 Cavendish Ave
> Devon Park SA 5008*
> *P: [phone]
> M: 0433 207 447
> F: **[phone]*
> *E: [hidden email]*
>